astro_prost.photoz_helpers

Attributes

default_model_path

default_dust_path

Functions

build_sfd_dir([file_path, data_dir])

Downloads directory of Galactic dust maps for extinction correction.

get_photoz_weights([file_path])

Get weights for MLP photo-z model.

ps1objidsearch(objid[, table, release, format, ...])

Do an object lookup by objid.

fetch_information_serially(url, data[, verbose, format])

A helper function called by serial_objid_search-- Queries PanStarrs API for data.

checklegal(table, release)

Checks if this combination of table and release is acceptable.

ps1search([table, release, format, columns, baseurl, ...])

Do a general search of the PS1 catalog (possibly without ra/dec/radius).

ps1metadata([table, release, baseurl])

Return metadata for the specified catalog and table. Snagged from the

post_url_serial(results, yse_id)

A helper function called by serial_objid_search.

serial_objid_search(objids[, table, release, columns, ...])

Given a list of ObjIDs, queries the PS1 server these object's Forced Mean Photometry,

get_common_constraints_columns()

Helper function that returns a dictionary of constraints used for the

preprocess(df[, path, ebv])

Preprocesses the data inside pandas.DataFrame object

load_lupton_model([model_path, dust_path])

Helper function that defines and loads the weights of our NN model and the output space of the NN.

evaluate(x, mymodel, range_z)

Evaluate the MLP for a set of PS1 inputs, and return predictions.

calc_photoz(hosts[, dust_path, model_path])

PhotoZ beta: not tested for missing objids.

get_photoz(df[, dust_path, model_path])

Evaluate photo-z model for Pan-STARRS forced photometry.

Module Contents

default_model_path = './MLP_lupton.hdf5'[source]
default_dust_path = '.'[source]
build_sfd_dir(file_path='./sfddata-master.tar.gz', data_dir=default_dust_path)[source]
Downloads directory of Galactic dust maps for extinction correction.

[Schlegel, Finkbeiner and Davis (1998)](http://adsabs.harvard.edu/abs/1998ApJ…500..525S).

Parameters:
  • fname (str) – Filename for dustmaps archive file.

  • data_path (str) – Target directory in which to extract ‘sfddata-master’ directory from archive file.

get_photoz_weights(file_path=default_model_path)[source]

Get weights for MLP photo-z model.

Parameters:

fname (str) – Filename of saved MLP weights.

ps1objidsearch(objid, table='mean', release='dr1', format='csv', columns=None, baseurl='https://catalogs.mast.stsci.edu/api/v0.1/panstarrs', verbose=False, **kw)[source]

Do an object lookup by objid.

Parameters:
  • objid (List of objids) – list of objids (or dictionary?)

  • table (str) – Can be 'mean', 'stack', or 'detection'.

  • release (str) – Can be ‘dr1’ or ‘dr2’.

  • format (str) – Can be ‘csv’, ‘votable’, or ‘json’

  • columns (arrray-like) – list of column names to include (None means use defaults)

  • baseurl (str) – base URL for the request

  • verbose (bool,optional) – print info about request

  • **kw (dictionary) – other parameters (e.g., ‘nDetections.min’:2)

fetch_information_serially(url, data, verbose=False, format='csv')[source]

A helper function called by serial_objid_search– Queries PanStarrs API for data.

Parameters:
  • url (str) – Remote PS1 url.

  • data (list) – List of objids requesting

  • verbose (bool, optional) – If True,

  • format (str) – Can be 'csv', 'json', or 'votable'.

Returns:

Return type:

str in format given by ‘format’.

checklegal(table, release)[source]
Checks if this combination of table and release is acceptable.

Raises a ValueError exception if there is problem.

Parameters:
  • table (str) – Table type. Can be 'mean', 'stack', or 'detection'

  • release (str) – The Pan-STARRS data release. Can be 'dr1' or 'dr2'.

Raises:

ValueError – Raises error if table and release combination are invalid.

ps1search(table='mean', release='dr1', format='csv', columns=None, baseurl='https://catalogs.mast.stsci.edu/api/v0.1/panstarrs', verbose=False, **kw)[source]

Do a general search of the PS1 catalog (possibly without ra/dec/radius).

Parameters:
  • table (str) – Table type. Can be 'mean', 'stack', or 'detection'

  • release (str) – The Pan-STARRS data release. Can be 'dr1' or ‘dr2'.

  • format (str) – Can be 'csv', 'votable', or 'json'.

  • columns (str) – Column names to include (None means use defaults).

  • baseurl (str) – Base URL for the request.

  • verbose (bool) – If true, print info about request.

  • **kw (dictionary) – Other parameters (e.g., 'nDetections.min':2). Note that this is required!

Returns:

Result of PS1 query, in 'csv', 'votable', or 'json' format.

Return type:

Same as ‘format’

ps1metadata(table='mean', release='dr1', baseurl='https://catalogs.mast.stsci.edu/api/v0.1/panstarrs')[source]
Return metadata for the specified catalog and table. Snagged from the

wonderful API at https://ps1images.stsci.edu/ps1_dr2_api.html.

Parameters:
  • table (str) – Table type. Can be 'mean', 'stack', or 'detection'

  • release (str) – The Pan-STARRS data release. Can be 'dr1' or 'dr2'.

  • baseurl (str) – Base URL for the request.

Returns:

Table with columns name, type, description.

Return type:

Astropy Table

post_url_serial(results, yse_id)[source]
A helper function called by serial_objid_search.

Post-processes the data retrieved from PS1 Servers into a pandas.DataFrame object.

Parameters:
  • results (str) – The string resulting from PS1 query.

  • yse_id (int) – local integer used for as an index tracking user objects vs retrived objects.

Returns:

DataFrame object of the retrieved data from PS1 servers

Return type:

pandas.DataFrame

Given a list of ObjIDs, queries the PS1 server these object’s Forced Mean Photometry,

then returns matches as a pandas.DataFrame.

Parameters:
  • objids (list) – list of PS1 objids for objects user would like to query

  • table (str) – Which table to perform the query on. Default ‘forced_mean’

  • release (str) – Which release to perform the query on. Default ‘dr2’

  • columns (list or None) – list of what data fields to include; None means use default columns. Default None

  • verbose (bool) – boolean setting level of feedback user received. default False

  • **constraints (dict) – Keyword dictionary with an additional constraints for the PS1 query

Returns:

list of pd.DataFrame objects. If a match was found, then the Dataframe contains data, else it only contains a local integer.

Return type:

pd.DataFrame

get_common_constraints_columns()[source]
Helper function that returns a dictionary of constraints used for the

matching objects in PS1 archive, and the columns of data we requre.

Returns:

dictionary with our constaint that we must have more than one detection

Return type:

dict

Returns:

List of PS1 fields required for matching and NN inputs

Return type:

list

preprocess(df, path='../data/sfddata-master/', ebv=True)[source]
Preprocesses the data inside pandas.DataFrame object

returned by serial_objid_search to the space of inputs of our Neural Network.

Parameters:
  • df (pandas DataFrame) – Dataframe object containing the data for each matched objid

  • path (str) – string path to extinction maps data

  • ebv (False) – boolean for lookup of extinction data. If False, all extinctions set to 0.

Returns:

Preprocessed inputs ready to be used as input to NN

Return type:

numpy ndarray

load_lupton_model(model_path=default_model_path, dust_path=default_dust_path)[source]

Helper function that defines and loads the weights of our NN model and the output space of the NN.

Parameters:
  • model_path (str) – path to the model weights.

  • dust_path (str) – path to dust map data files.

Returns:

Trained photo-z MLP.

Return type:

tensorflow keras Model

Returns:

Array of binned redshift space corresponding to the output space of the NN

Return type:

numpy ndarray

evaluate(x, mymodel, range_z)[source]

Evaluate the MLP for a set of PS1 inputs, and return predictions.

Parameters:
  • x (array-like) – PS1 properties of associated hosts.

  • mymodel (tensorflow keras Model) – MLP model for photo-z estimation.

  • range_z (array-like) – Grid over which to evaluate the posterior distribution of photo-zs.

Returns:

Posterior distributions for the grid of redshifts defined as `np.linspace(0, 1, n)`

Return type:

numpy ndarray shape of (df.shape[0], n)

Returns:

means

Return type:

numpy ndarray shape of (df.shape[0],)

Returns:

Standard deviations

Return type:

numpy ndarray shape of (df.shape[0],)

calc_photoz(hosts, dust_path=default_dust_path, model_path=default_model_path)[source]
PhotoZ beta: not tested for missing objids.

photo-z uses a artificial neural network to estimate P(Z) in range Z = (0 - 1) range_z is the value of z posterior is an estimate PDF of the probability of z point estimate uses the mean to find a single value estimate error is an array that uses sampling from the posterior to estimate a std dev. Relies upon the sfdmap package, (which is compatible with both unix and windows), found at https://github.com/kbarbary/sfdmap.

Parameters:

hosts (pandas DataFrame) – The matched hosts from GHOST.

Returns:

The matched hosts from GHOST, with photo-z point estimates and uncertainties.

Return type:

pandas DataFrame

get_photoz(df, dust_path=default_dust_path, model_path=default_model_path)[source]

Evaluate photo-z model for Pan-STARRS forced photometry.

Parameters:
  • df (pandas DataFrame) – Pan-STARRS forced mean photometry data, you can get it using `ps1objidsearch` from this module, Pan-STARRS web-portal or via astroquery i.e., `astroquery.mast.Catalogs.query_{criteria,region}(…, catalog='Panstarrs',table='forced_mean')`

  • dust_path (str) – Path to dust map data files

  • model_path (str) – path to the data file with weights for MLP photo-z model

Returns:

Posterior distributions for the grid of redshifts defined as `np.linspace(0, 1, n)`

Return type:

numpy ndarray shape of (df.shape[0], n)

Returns:

means

Return type:

numpy ndarray shape of (df.shape[0],)

Returns:

Standard deviations

Return type:

numpy ndarray shape of (df.shape[0],)