The Right Stuff: Downloading a Petavoxel

Working with petavoxel datasets requires careful consideration of your finances, time, equipment, and patience. The below charts may help focus your efforts and attention. Bandwidth costs can become substantial, though providers offer discount packages of varying utility. Note that these numbers may not be reachable due to processor time spent in decompression.

Table 1. Days to Download 1 PB This table demonstrates the feasibility of downloading a petabyte of image data at four different resolutions scaled down by 2x2 using commonly available bandwidth configurations. 100 Mbps and 1 Gbps are typical of home or workstation connections, 5, 10, and 40 Gbps are more typical of specialized setups or entire buildings, 100 Gbps is usually reserved for specialized high throughput applications or internet infrastructure. Mip 0 is the highest resolution image, with each successive mip level having half the resolution of the previous level along both the width and height of the image. Mip 3 is a near-isotropic resolution that is useful for generating meshes and skeletons. Mips 1 and 2 can sometimes be used for alignment or automated segmentation of images.

In order to make use of images casually, download times should take less than a day. At the outside limits of patience, most people would give up after one or two weeks unless they were highly motivated. Even highly motivated people would probably look for another avenue if a download were to take longer than a month. Given a 100 Mbps home internet connection only mip 3 is feasible. Using gigabit internet, mip 1 is feasible. Starting around 2 Gbps, mip 3 becomes casually accessible. At 3 Gbps, the highest resolution image becomes attainable for someone highly motivated. It becomes practical to process the highest resolution image starting slightly under 10 Gbps and casual to do so at 100 Gbps.

Table 2. Algorithm Efficiency Requirements for Streaming In order to take advantage of the timelines cited in Table 1, at least conceptually, streaming processing is required. The table shows that as the network bandwidth increases, the code efficiency target rises. As the number of parallel cores increase, the required efficiency decreases proportionately. (blue) configurations common to home offices or laboratory workstations (light teal) target for the least powerful setup that can casually process mip 3 (teal) target for the least powerful setup able to practically process full resolution images (red) these targets surpass the abilities of non-overclocked CPUs (light red) these targets are very difficult to hit, as they are nearing the clock speed of commonly available processors.