DownloadManager

class halotools.sim_manager.DownloadManager

Bases: object

Class used to scrape the web for simulation data and cache the downloaded catalogs.

For a list of available pre-processed halo catalogs provided by Halotools, see Simulations and Halo Catalogs Provided by Halotools.

Methods Summary

download_processed_halo_table(simname, ...)	Method to download one of the pre-processed binary files storing a reduced halo catalog.
download_ptcl_table(simname, redshift[, ...])	Method to download one of the binary files storing a random downsampling of dark matter particles.

Methods Documentation

download_processed_halo_table(simname, halo_finder, redshift, dz_tol=0.1, overwrite=False, version_name='halotools_v0p4', download_dirname='std_cache_loc', ignore_nearby_redshifts=False, **kwargs)

Method to download one of the pre-processed binary files storing a reduced halo catalog.

Parameters:

simnamestring

Nickname of the simulation. Currently supported simulations are Bolshoi (simname = bolshoi), Consuelo (simname = consuelo), MultiDark (simname = multidark), and Bolshoi-Planck (simname = bolplanck).

halo_finderstring

Nickname of the halo-finder, e.g. rockstar or bdm.

redshiftfloat

Redshift of the requested snapshot. Must match one of theavailable snapshots within dz_tol, or a prompt will be issued providing the nearest available snapshots to choose from.

version_namestring, optional

Nickname of the version of the halo catalog used to differentiate between the same halo catalog processed in different ways. Default is set in sim_defaults.

download_dirnamestr, optional

Absolute path to the directory where you want to download the catalog. Default is std_cache_loc, which will store the catalog in the following directory: $HOME/.astropy/cache/halotools/halo_tables/simname/halo_finder/

dz_tolfloat, optional

Tolerance value determining how close the requested redshift must be to some available snapshot before issuing a warning. Default value is 0.1.

overwriteboolean, optional

If a file with the same filename already exists in the requested download location, the overwrite boolean determines whether or not to overwrite the file. Default is False, in which case no download will occur if a pre-existing file is detected.

ignore_nearby_redshiftsbool, optional

Flag used to determine whether nearby redshifts in cache will be ignored. If there are existing halo catalogs in the Halotools cache with matching simname, halo_finder and version_name, and if one or more of those catalogs has a redshift within dz_tol, then the ignore_nearby_redshifts flag must be set to True in order for the new halo catalog to be stored in cache. Default is False.

Notes

If after downloading the catalog you decide that you want to move it to a new location on disk, you will need to be sure your cache directory is informed of the relocation. In this case, see Relocating Simulation Data and Updating the Cache for instructions.

Examples

>>> from halotools.sim_manager import sim_defaults
>>>
>>> dman = DownloadManager()
>>> simname = 'bolplanck'
>>> z = 2
>>> version_name = sim_defaults.default_version_name
>>> halo_finder = sim_defaults.default_halo_finder
>>> dman.download_processed_halo_table(simname = 'bolplanck', halo_finder = halo_finder, version_name = version_name, redshift = z) 

Now that you have downloaded the catalog, it is stored in the default cache location:

$HOME/.astropy/cache/halotools/halo_catalogs/ Use the download_dirname keyword argument to store the catalog in an alternate location. Wherever you store it, after calling the download_processed_halo_table method you can load the catalog into memory as follows:

>>> from halotools.sim_manager import CachedHaloCatalog
>>> halocat = CachedHaloCatalog(simname = 'bolplanck', redshift = z) 

Since you chose default values for the version_name and halo_finder, it is not necessary to specify these keyword arguments. The halocat has metadata attached to it describing the simulation, snapshot, catalog processing notes, etc. The actual halos are stored in the form of an Astropy Table data structure and can be accessed as follows:

>>> halos = halocat.halo_table 
>>> array_of_masses = halos['halo_mvir'] 
>>> array_of_x_position = halos['halo_x'] 

download_ptcl_table(simname, redshift, dz_tol=0.1, overwrite=False, version_name='halotools_v0p4', download_dirname='std_cache_loc', ignore_nearby_redshifts=False, **kwargs)

Method to download one of the binary files storing a random downsampling of dark matter particles.

Parameters:

simnamestring

Nickname of the simulation. Currently supported simulations are Bolshoi (simname = bolshoi), Consuelo (simname = consuelo), MultiDark (simname = multidark), and Bolshoi-Planck (simname = bolplanck).

redshiftfloat

Redshift of the requested snapshot. Must match one of theavailable snapshots within dz_tol, or a prompt will be issued providing the nearest available snapshots to choose from.

version_namestring, optional

Nickname of the version of the halo catalog used to differentiate between the same halo catalog processed in different ways. Default is set in sim_defaults.

download_dirnamestr, optional

Absolute path to the directory where you want to download the catalog. Default is std_cache_loc, which will store the catalog in the following directory: $HOME/.astropy/cache/halotools/halo_tables/simname/halo_finder/

dz_tolfloat, optional

Tolerance value determining how close the requested redshift must be to some available snapshot before issuing a warning. Default value is 0.1.

overwriteboolean, optional

If a file with the same filename already exists in the requested download location, the overwrite boolean determines whether or not to overwrite the file. Default is False, in which case no download will occur if a pre-existing file is detected.

ignore_nearby_redshiftsbool, optional

Flag used to determine whether nearby redshifts in cache will be ignored. If there are existing halo catalogs in the Halotools cache with matching simname, halo_finder and version_name, and if one or more of those catalogs has a redshift within dz_tol, then the ignore_nearby_redshifts flag must be set to True in order for the new halo catalog to be stored in cache. Default is False.

Notes

If after downloading the catalog you decide that you want to move it to a new location on disk, you will need to be sure your cache directory is informed of the relocation. In this case, see Relocating Simulation Data and Updating the Cache for instructions.

Examples

>>> dman = DownloadManager()
>>> simname = 'bolplanck'
>>> z = 2
>>> version_name = sim_defaults.default_version_name
>>> dman.download_ptcl_table(simname = 'bolplanck', version_name = version_name, redshift = z) 

Now that you have downloaded the particles, the data is stored in the default cache location:

$HOME/.astropy/cache/halotools/particle_catalogs/ Use the download_dirname keyword argument to store the catalog in an alternate location. Wherever you store it, after calling the download_ptcl_table method you can access particle data by loading the associated halo catalog into memory:

>>> from halotools.sim_manager import CachedHaloCatalog
>>> halocat = CachedHaloCatalog(simname = 'bolplanck', redshift = z, halo_finder = 'rockstar') 

Since you chose default values for the version_name, it is not necessary to specify that keyword arguments. The halocat has metadata attached to it describing the simulation, snapshot, catalog processing notes, etc. The actual particles are stored in the form of an Astropy Table data structure and can be accessed as follows:

>>> particles = halocat.ptcl_table 
>>> array_of_x_position = particles['x']