Space Sciences/Astrophysics Seminar with Neven Caplar on Vera C. Rubin Observatory and LINCC: more data, more software, more science

We are nearing completion and first light for the Vera C. Rubin Observatory. The survey conducted will provide unprecedented depth and width of synoptic data. For example, Rubin's Year #1 dataset will be O(100TB) in size. Processing, calibrating, and delivering such a large amount of data in a valuable way for scientists is a significant challenge for the data management team. It is critical for the scientific output of the survey. At the start of the talk I will discuss the current status of the observatory, the near-term plans, and how scientists can already engage with the data. I will then introduce LINCC Frameworks; an ambitious program to develop state-of-the-art analysis techniques capable of meeting the scale and complexity demanded by the data produced by the Vera C. Rubin Observatory. I will introduce LSDB, a software package that facilitates large-scale analytics across multiple datasets. LSDB's efficacy lies in its ability to shard expansive datasets and index sources within the Healpix space, leveraging catalog density per index. LSDB seamlessly integrates with the TAPE project for time-domain data analysis, enabling users to parallelize calculations through the DASK framework. Towards the end, I will also present an example use case of searching for rare events and anomalies in existing datasets and showcase how it can be extended to Rubin-size datasets. Finally, LINCC Frameworks also sponsors incubators, three-month programs to scale up existing software projects. I will discuss how we can help you with your projects and explain the opportunities to engage with LSDB/TAPE.

Sponsored by the McDonnell Center for the Space Sciences.