Singularity
An end-to-end solution for data onboarding to Filecoin.
Facilitating petabyte-scale data ingestion for storage clients and providers.
Features
Simple data processing
Singularity automates data preparation by automatically chunking your data and generating CAR files.
Configurable and modular
Pick-and-choose the components you want. Process local data or point to a public S3 bucket.
Robust deal execution
Customize the deal execution strategy: deal with your preferred storage providers with a schedule that works.
Indexing and retrieval
Data is automatically indexed on IPFS. Retrievals are done with a simple command.
Singularity is an open-source tool designed by @Xinan Xu and @Fei Yan and maintained by the Data Onboarding team at Protocol Labs. Our goal is to develop tools that help data owners and storage providers ingest massive amounts of data to the Filecoin network.
It is a easy-to-use NPM package that works with Lotus and Boost to tie together every step of data onboarding.
How it works
1. Prepare the data
The data preparation module will scan and convert a local folder recursively into CAR files ready to be onboarded to Filecoin Network. Singularity also indexes your files on IPFS for easy retrieval.
2. Distribute the CAR files
Easily host an HTTP server with the CAR files to be downloaded by the storage provider. You can also send the CAR files offline if the files are too large.
3. Pick your storage providers
Use a solution for finding storage providers or reach out on Slack. Most storage providers today are interested in taking verified deals, so you need to familiarize yourself with Filecoin Plus.
4. Make storage deals
Leverage Lotus and Boost to automatically make deals with the storage providers of your choice.
Made with love by Protocol Labs