timewise.wise_bigdata_desy_cluster.WISEDataDESYCluster

class timewise.wise_bigdata_desy_cluster.WISEDataDESYCluster(base_name, parent_sample_class, min_sep_arcsec, n_chunks, clean_outliers_when_binning=True, multiply_flux_error=True)

A class to download WISE data with multiple threads and do the binning on the DESY cluster. In addition to the attributes of WiseDataByVisit this class has the following attributes:

Parameters:

• executable_filename (Path) – the filename of the executable that will be submitted to the cluster

• submit_file_filename (Path) – the filename of the submit file that will be submitted to the cluster

• job_id (str) – the job id of the submitted job

• cluster_jobID_map (dict) – a dictionary mapping the chunk number to the cluster job id

• clusterJob_chunk_map (dict) – a dictionary mapping the cluster job id to the chunk number

• cluster_info_file (Path) – the filename of the file that stores the cluster info, loaded by the cluster jobs

• start_time (float) – the time when the download started

__init__(base_name, parent_sample_class, min_sep_arcsec, n_chunks, clean_outliers_when_binning=True, multiply_flux_error=True)

Constructor of the class.

Parameters:

• base_name (str) – the base name of the data directory

• parent_sample_class (ParentSampleBase) – the parent sample class

• min_sep_arcsec (float) – query region around source for positional query

• n_chunks (int) – number of chunks to split the sample into

• clean_outliers_when_binning (bool) – if True, clean outliers when binning

Methods

__init__(base_name, parent_sample_class, ...)	Constructor of the class.
add_flux_densities_to_saved_lightcurves(service)	Adds flux densities to all downloaded lightcurves
add_flux_density(lightcurve, mag_key, ...[, ...])	Adds flux densities to a lightcurves
add_luminosity_to_saved_lightcurves(service)	Add luminosities to all lightcurves, calculated from flux densities and distance or redshift
bin_lightcurve(lightcurve)	Combine the data by visits of the satellite of one region in the sky.
calculate_epochs(f, e, visit_mask, counts, ...)	Calculates the binned epochs of a lightcurve.
calculate_metadata(service[, chunk_number, ...])	Calculates the metadata for all downloaded lightcurves.
calculate_metadata_single(lc)	Calculates some metadata, describing the variability of the lightcurves.
calculate_position_mask(lightcurve, ra, dec, ...)	Estimated the 90th percentile of the angular separations from the given position.
clear_cluster_log_dir()	Clears the directory where cluster logs are stored
collect_condor_status()	Gets the condor status and saves it to private attribute
condor_status(job_id)	Get the status of jobs running on condor.
dump_tap_cache()
find_color_correction(w1_minus_w2)	Find the color correction based on the W1-W2 color.
get_condor_status()	Queries condor to get cluster status.
get_coverage(chunk, lum_key[, ...])	Get the coverage of the MEASURED median for a given chunk and lum_key
get_db_name(table_name[, nice])	Get the right table name
get_photometric_data([tables, perc, ...])	Load photometric data from the IRSA server for the matched sample.
get_position_mask(service, chunk_number)	Get the position mask for a chunk
get_quantiles_label(df[, cl])	Get the quantiles label for a given coverage level
get_red_chi2(chunk, lum_key[, use_bigdata_dir])	Get the reduced chi2 for a given chunk or multiple chunks
get_sample_photometric_data([max_nTAPjobs, ...])	An alternative to get_photometric_data() that uses the DESY cluster and is optimised for large datasets.
get_submit_file_filename(ids)	Get the filename of the submit file for given job ids
get_unbinned_lightcurves(chunk_number[, clear])	Get the unbinned lightcurves for a given chunk number.
get_visit_map(lightcurve)	Create a map datapoint to visit
load_data_product(service[, chunk_number, ...])	Load data product from disk
load_tap_cache()
luminosity_from_flux_density(flux_density, band)	Converts a flux density into a luminosity
make_chi2_plot([index_mask, chunks, ...])	Make a plot of the reduced chi2 distribution for a given chunk or multiple chunks
make_coverage_plots([index_mask, chunks, ...])	Make the coverage plots for the measured median of the specified luminosity unit
make_executable_file()	Produces the executable that will be submitted to the NPX cluster.
make_submit_file(job_ids[, node_memory, ...])	Produces the submit file that will be submitted to the NPX cluster.
match_all_chunks([table_name, ...])	Match the parent sample to a WISE catalogue and add the result to the parent sample.
plot_diagnostic_binning(service, ind[, ...])	Show a skymap of the single detections and which bin they belong to next to the binned lightcurve
plot_lc(parent_sample_idx[, service, ...])	Make a pretty plot of a lightcurve
run_cluster(node_memory, service)	Run the DESY cluster
submit_to_cluster(node_memory[, ...])	Submit jobs to cluster
vegamag_to_flux_density(vegamag, band[, ...])	This converts the detector level brightness m in Mag_vega to a flux density F
wait_for_job([job_id])	Wait until the cluster job is done

Attributes

BASHFILE
Npoints_key
active_tap_phases
aperture_corrections
band_plot_colors
band_wavelengths
bands
chunk_map
constraints
done_tap_phases
env_file
error_key_ext
flux_density_key_ext
flux_error_factor
flux_key_ext
luminosity_key_ext
mag_key_ext
magnitude_zeropoints
magnitude_zeropoints_corrections
mean_key
median_key
n_chunks
n_cluster_jobs_per_chunk
parent_sample
parent_sample_wise_skysep_key
parent_wise_source_id_key
photometry_table_keymap
query_types
rms_key
running_tap_phases
service
service_url
status_cmd
table_names
tap_cache_filenames
upper_limit_key
zeropoint_key_ext