brain.space's 5 researchers used to wait 1-2 days to debug pipelines over their EEG, neuroimaging, and ECG data. Their Flyte + Spark + SQL stack put a data engineer between every researcher and their own data. DataChain replaces that gatekeeper with typed datasets, lineage, processing history, and distributed compute the researchers run themselves. brain.space cut pipeline runtime 5-10× as a result.
Same SDK from Python, Jupyter, or Claude Code / Cursor / Codex.
Works with your stack
How it works
Point at S3, GCS, Azure, Nebius, or on-prem. Files stay where they are; DataChain reads schemas, lineage, and processing history.
From a notebook, a script, or an AI agent. Dispatch to 1 to 1000+ machines in your VPC when scale demands it.
Every pipeline saves a typed dataset to the team's registry. The next session reads what exists instead of re-deriving from raw bytes.
🤝 Collaboration: data work that compounds
Bio research is a team sport: researchers, data engineers, hardware, QA, sometimes external partners. Without a shared layer, every handoff is an export, a copy, or a “where's that EEG cohort from January?” Slack ping. DataChain makes each dataset a versioned, named, lineage-traced reference anyone on the team can read.
- Central registry: browse datasets by name, version, schema, and author.
- Lineage answers “how was this produced?” without pinging the original researcher.
- Sharing is a dataset reference, not a CSV export.
- Bytes never leave storage; access governed by your existing IAM.
“What surprised me was how easily researchers adopted DataChain, data tools are usually hard for non-engineers. What surprised me more was when hardware and QA started asking for access too.”
Versioned datasets are the team's shared reference. Every pipeline run grows the layer.
📊 Visualization: see your data in storage
Researchers inspect DICOM, NIfTI, and EEG traces inline in Studio. Every file links to its dataset version, schema, and the code that produced it, so a debate over a scan and the code stay in one place.
- Explore DICOM, NIfTI, and other biomedical files in the UI.
- Validate preprocessing and feature extraction assumptions early; inspect neural and physiological patterns inline.
- Compare versions of the same scan or signal side-by-side from the central registry.
Zero exports. One source of truth.
🦾 Distributed compute the whole team can use
A researcher's laptop Python session can't spin up 300 CPUs. With DataChain, the same notebook, script, or agent dispatches jobs across hundreds of machines in your VPC: 300 CPUs on GCP, 10 T4 GPUs on GCP, 4 A100 GPUs on Nebius.
- Attach multiple clusters: CPU pools, GPU pools, high-memory pools.
- Scale from 1 to 1000+ machines from the SDK, no extra framework.
- DataChain manages the clusters for you. Workers spin up on demand, spin down when idle; you don't forget the 20 GPUs running over the weekend.
- Bytes never leave your storage. BYOC by default: compute runs in your AWS, GCP, or Azure account, behind your VPC, under your existing IAM.
Your scripts and agents become research engineers with infrastructure access.
🤖 Where Claude Code actually works on data
Software teams 10× with Claude Code, Cursor, and Codex. Data teams don't: agents hallucinate schemas, re-extract on every run, and stay stuck on the laptop. DataChain gives them the typed catalog, lineage, and clusters they were missing.
- Shared: every agent-run pipeline lands in the team's registry, with source code, parameters, inputs, and author attached.
- Code generation for data: vanilla agents can't write high-scale data jobs, and 0% of their tasks materialize a reusable dataset. DataChain agents produce typed datasets that scale across clusters; cost-of-failure drops 2.7× (9× on image work), and follow-up tasks run 8.4× faster, 3.4× cheaper.
- Distributed jobs: an agent runs a data job on 30 machines to crunch your medical data.
Claude Code, finally productive on data.
Production-grade compliance
Subject data never leaves your cloud account. DataChain is SOC2-certified and deploys as BYOC: compute and storage stay in your AWS, GCP, or Azure, under your IAM. Every signal-to-feature transformation is traceable; every dataset version immutable. Subject withdrawal becomes a query across the lineage graph; IRB amendments propagate as new dataset versions. brain.space runs the whole stack across their own AWS and Nebius (Nebius for GPUs).
Audit-ready by default.
brain.space: 5-10× on EEG and neuroimaging work
brain.space's Brain Data-as-a-Service platform converts EEG, neuroimaging, ECG, EDA, and eye-tracking signals into reusable datasets that feed Large Mental Models. Before DataChain, their 5 researchers waited 1-2 days to debug pipelines and couldn't run distributed compute without a data engineer.
“We realized we were solving a problem we shouldn't be solving. With DataChain, what used to require data engineers is now handled seamlessly by researchers, and the whole team moved to the next level.”
With DataChain in place:
- All 5 researchers run pipelines themselves, no data-engineering bottleneck.
- Debugging from 1-2 days to minutes through checkpoint recovery and immutable lineage.
- 5-10× faster overall processing. High-frequency EEG dropped from days to hours.
- BYOC compliance kept intact. Compute runs across brain.space's own AWS and Nebius (GPUs); DataChain orchestrates without touching the bytes.
brain.space proves the model: typed datasets, shared lineage, and distributed compute lift every researcher, with or without AI agents.
What changes when you switch from Flyte + Spark + SQL
brain.space's actual migration. The same shape generalizes to any Flyte / Spark / Airflow / custom-ETL stack on multimodal bio data.
Frequently asked
No. DataChain runs as BYOC: compute and storage stay in your AWS, GCP, Azure, Nebius, or on-prem account. The control plane orchestrates; the data plane stays yours.
SOC2. For HIPAA, GxP, IRB, GDPR, or any other framework your team operates under, BYOC means your existing controls carry through unchanged.
It's Python they already know. brain.space's 5 researchers were running pipelines themselves within weeks of adopting DataChain.
DataChain gives DE a typed, versioned substrate to operate on, and gives researchers self-service access on top. DE shifts from gatekeeper to enabler: they build the pipelines researchers run and extend.
Subject withdrawal becomes a query across the lineage graph: find every derived dataset that touched the subject. IRB amendments propagate as new dataset versions, keeping the audit trail intact.