Flexible Deal Making
Deal with your preferred storage providers with a schedule that works for you.
Seamlessly onboard PiB scale data to the Filecoin storage network
Singularity is an open source, all-in-one tool for data onboarding, including data preparation, robust deal execution, built-in data indexing and retrieval.
- IDTypePathScanningStateDagGenState
- 1s3foldingathome-covid19completecreated
- 2s3multi-token-completionreadycreated
- 3s3fast-ai-nlpreadycreated
- 4s3nasanexcompletecreated
- 5gcsterrafusionreadycreated
- 6s3conversational-searchreadycreated
- 7s3noaa-nexradcompletecreated
- 8s3usgs-landsatreadycreated
- 9s3brazil-data-cubesreadycreated
- 10gcsnduicompletecreated
- 11koofrjanelia-flylightreadycreated
- 12s3commoncrawlreadycreated
WHY SINGULARITY?
Efficient, secure, and flexible data onboarding.
Basic data management isn’t enough. With your needs in mind Singularity is committed to improving the experience of data on-boarding.
Simple Data Processing
Singularity seamlessly processes data from your local sources or public S3 buckets and automates data preparation
Data Integrity
Singularity fosters decentralization by allowing users minimal reliance on storage providers, facilitated by easy verification and support from Filecoin protocol
Robust File Management
Supports file versioning, deletion, and dataset folder structure exploration
Features
Inline Data Preparation
We value your storage space. Our unique inline preparation feature eliminates the need for extra storage for CAR files, instead maintaining a metadata database that maps CAR files to the original data source.
Versatile Data Source Integration
With Singularity, you're not bound by storage types. We offer seamless integration with over 40 different storage services, ranging from popular consumer products like Dropbox and Google Drive to enterprise solutions such as AWS S3, Azure Blob Storage, FTP, and HDFS.
Flexible Deal Making
Singularity puts you in control. Bring your own list of storage providers and define your deal-making process, whether it’s a one-off batch or a scheduled cron. Storage providers can also propose deals, giving you ultimate flexibility.
Effortless Deal Tracking
Deal tracking is to track the status of each deal proposal. Singularity helps:
- Validate how each file or deal is distributed with Filecoin storage providers
- Auto renewal for expired or slashed deals
Indexing and Retrieval
Singularity as a retrieval provider with a simple command. Each file is retrievable using File CID or or File Path.
Smart Wallet Management
Deal with your preferred storage providers with a schedule that works for you.
How it works
01 Prepare the Data
The data preparation module will scan and convert a local folder recursively into CAR files ready to be onboarded to Filecoin Network. Singularity also indexes your files on IPFS for easy retrieval.
02 Distribute CAR Files
Easily host an HTTP server with the CAR files to be downloaded by the storage provider. You can also send the CAR files offline if the files are too large.
03 Pick your Storage Providers
Bring your own preferred storage providers, or join Filecoin slack channel to find reputable providers. Most storage providers today are interested in taking verified deals.
04 Make Storage Deals
The client sets a schedule to send deals to storage providers through Singularity, either once or on a recurring basis.
Roadmap
September
2023
Milestone 1
Release S3 compatible API (Alpha)
Milestone 2
Interface with one simple API that handles:
- Data Preparation
- Deal-making
- Tracking deal status
October
2023
Milestone 3
Web UI for Singularity (Alpha): Does not require knowing Filecoin specifics but rather an easy drap and drop file upload experience
November
2023
Milestone 4
S3 compatible API (Beta): A simple set of API’s (POST/GET/GET-STATUS) to be used by an independent software vendor (ISV) to store and retrieve data from Filecoin and track status of data stored
December
2023
Milestone 5
Automate the process to find storage providers by integrating Singularity with Spade
Keep up