Transfer data¶
This guide shows how to transfer data from a source database into the currently connected database.
# pip install lamindb
!lamin init --storage ./test-transfer --modules bionty
Show code cell output
! not connected, call: ln.connect('account/name')
! using anonymous user (to identify, call: lamin login)
→ initialized lamindb: anonymous/test-transfer
import lamindb as ln
ln.track("ITeOtm7bhtdq")
Show code cell output
→ connected lamindb: anonymous/test-transfer
→ created Transform('ITeOtm7bhtdq0000', key='transfer.ipynb'), started new Run('E1tjoFgeKUV0SeIu') at 2025-12-01 17:01:28 UTC
→ notebook imports: lamindb==1.17a1
Query all artifacts in the laminlabs/lamindata instance and filter them to their latest versions.
# query all latest artifact versions
artifacts = ln.Artifact.connect("laminlabs/lamindata").filter(is_latest=True)
# convert the QuerySet to a DataFrame and show the latest 5 versions
artifacts.to_dataframe().head()
Show code cell output
| uid | key | description | suffix | kind | otype | size | hash | n_files | n_observations | params | version | is_latest | is_locked | created_at | branch_id | space_id | storage_id | run_id | schema_id | created_by_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||||||
| 12380 | YIQg5lsdHEnYNxL60001 | samplesheet.csv | None | .csv | None | DataFrame | 502.0 | 97v5_3h9UQnTIN5UeBYAdA | NaN | NaN | None | None | True | False | 2025-11-25 16:54:18.033021+00:00 | 1 | 1 | 2 | None | None | 2 |
| 12374 | OHg4CS0OCErxUXKG0000 | None | Output artifact for run 1483 | .json | None | None | 80.0 | jC-6xYWK_Kql82E7RSVAEw | NaN | NaN | None | None | True | False | 2025-11-21 12:29:14.708445+00:00 | 1 | 1 | 2 | 1483 | None | 18 |
| 12373 | W7jXMy7nLINQwD7e0000 | None | Output artifact for run 1482 | .json | None | None | 80.0 | YLtlxHMWj2cL3O50WuNjGw | NaN | NaN | None | None | True | False | 2025-11-21 12:29:06.575178+00:00 | 1 | 1 | 2 | 1482 | None | 18 |
| 12372 | ERCIUR3bgOOy6uxK0000 | examples/dataset2.h5ad | None | .h5ad | dataset | AnnData | 26896.0 | RKJjWbINYNIwYU8BxCejMw | NaN | 3.0 | None | None | True | False | 2025-11-20 11:04:08.172526+00:00 | 1 | 1 | 2 | None | 151 | 6 |
| 12371 | zos9ajq0y1dXAy8Z0000 | None | Output artifact for run 1479 | .json | None | None | 80.0 | 6HZLHaB61cjU5TAeunfgwg | NaN | NaN | None | None | True | False | 2025-11-20 10:18:07.310369+00:00 | 1 | 1 | 2 | 1479 | None | 18 |
You can now further subset or search the QuerySet. Here we query by whether the description contains “tabula sapiens”.
artifact = artifacts.filter(description__contains="Tabula Sapiens").first()
artifact.describe()
Show code cell output
Artifact: tabula_sapiens_lung.h5ad (6Wph) | description: Part of Tabula Sapiens, a benchmark, first-draft human cell atlas. ├── uid: dPraor9rU1EofcFb6Wph run: IeeCQQO (ux-session-tb-lung) │ hash: 8mB1KK2wd51F6HQdvqipcQ size: 3.6 GB │ branch: main space: all │ created_at: 2023-07-14 19:00:30 UTC created_by: Koncopd ├── storage/path: s3://lamindata/tabula_sapiens_lung.h5ad └── Labels └── .ulabels ULabel TSP1, TSP2, TSP14 .tissues bionty.Tissue lung .cell_types bionty.CellType type I pneumocyte, adventitial cell, ba… .experimental_factors bionty.ExperimentalFactor anoxya, stroke
By saving the artifact record that’s currently attached to the source database instance, you transfer it to the default database instance.
artifact.save()
Show code cell output
→ transferred: Artifact(uid='dPraor9rU1EofcFb6Wph'), Storage(uid='D9BilDV2')
Artifact(uid='dPraor9rU1EofcFb6Wph', version=None, is_latest=True, key='tabula_sapiens_lung.h5ad', description='Part of Tabula Sapiens, a benchmark, first-draft human cell atlas.', suffix='.h5ad', kind=None, otype=None, size=3899435772.0, hash='8mB1KK2wd51F6HQdvqipcQ', n_files=None, n_observations=None, params=None, branch_id=1, space_id=1, storage_id=2, run_id=2, schema_id=None, created_by_id=1, created_at=2023-07-14 19:00:30 UTC, is_locked=False)
How do I know if a record is saved in the default database instance or not?
Every record has an attribute ._state.db which can take the following values:
None: the record has not yet been saved to any database"default": the record is saved on the default database instance"account/name": the record is saved on a non-default database instance referenced byaccount/name(e.g.,laminlabs/lamindata)
The artifact record has been transferred to the current database without feature & label annotations, but with updated data lineage.
artifact.describe()
Show code cell output
Artifact: tabula_sapiens_lung.h5ad (6Wph) | description: Part of Tabula Sapiens, a benchmark, first-draft human cell atlas. ├── uid: dPraor9rU1EofcFb6Wph run: 8Kxl2LV (__lamindb_transfer__/4XIuR0tvaiXM) │ hash: 8mB1KK2wd51F6HQdvqipcQ size: 3.6 GB │ branch: main space: all │ created_at: 2023-07-14 19:00:30 UTC created_by: anonymous └── storage/path: s3://lamindata/tabula_sapiens_lung.h5ad
You see that the data itself remained in the original storage location, which has been added to the current instance’s storage location as a read-only location (indicated by the fact that the instance_uid doesn’t match the current instance).
ln.Storage.to_dataframe()
Show code cell output
| uid | root | description | type | region | instance_uid | is_locked | created_at | branch_id | space_id | created_by_id | run_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||
| 2 | D9BilDV2 | s3://lamindata | None | s3 | us-east-1 | 4XIuR0tvaiXM | False | 2023-04-22 05:50:06.537267+00:00 | 1 | 1 | 1 | 2.0 |
| 1 | HJYSDnG3pb5f | /home/runner/work/lamindb/lamindb/docs/test-tr... | None | local | None | 1FHu5eE0uxm4 | False | 2025-12-01 17:01:24.250000+00:00 | 1 | 1 | 1 | NaN |
See the state of the database.
ln.view()
Show code cell output
****************
* module: core *
****************
Artifact
| uid | key | description | suffix | kind | otype | size | hash | n_files | n_observations | params | version | is_latest | is_locked | created_at | branch_id | space_id | storage_id | run_id | schema_id | created_by_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||||||
| 1 | dPraor9rU1EofcFb6Wph | tabula_sapiens_lung.h5ad | Part of Tabula Sapiens, a benchmark, first-dra... | .h5ad | None | None | 3899435772 | 8mB1KK2wd51F6HQdvqipcQ | None | None | None | None | True | False | 2023-07-14 19:00:30.621330+00:00 | 1 | 1 | 2 | 2 | None | 1 |
Run
| uid | name | started_at | finished_at | params | reference | reference_type | is_locked | created_at | branch_id | space_id | transform_id | report_id | _logfile_id | environment_id | created_by_id | initiated_by_run_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||
| 2 | 8Kxl2LVhjpivnXVv | None | 2025-12-01 17:01:31.112000+00:00 | None | None | None | None | False | 2025-12-01 17:01:31.112000+00:00 | 1 | 1 | 2 | None | None | None | 1 | 1.0 |
| 1 | E1tjoFgeKUV0SeIu | None | 2025-12-01 17:01:28.441390+00:00 | None | None | None | None | False | 2025-12-01 17:01:28.442000+00:00 | 1 | 1 | 1 | None | None | None | 1 | NaN |
Storage
| uid | root | description | type | region | instance_uid | is_locked | created_at | branch_id | space_id | created_by_id | run_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||
| 2 | D9BilDV2 | s3://lamindata | None | s3 | us-east-1 | 4XIuR0tvaiXM | False | 2023-04-22 05:50:06.537267+00:00 | 1 | 1 | 1 | 2.0 |
| 1 | HJYSDnG3pb5f | /home/runner/work/lamindb/lamindb/docs/test-tr... | None | local | None | 1FHu5eE0uxm4 | False | 2025-12-01 17:01:24.250000+00:00 | 1 | 1 | 1 | NaN |
Transform
| uid | key | description | type | source_code | hash | reference | reference_type | config | is_flow | version | is_latest | is_locked | created_at | branch_id | space_id | flow_id | environment_id | created_by_id | _template_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||||||||||
| 2 | 4XIuR0tvaiXM0000 | __lamindb_transfer__/4XIuR0tvaiXM | Transfer from `laminlabs/lamindata` | function | None | None | None | None | None | False | None | True | False | 2025-12-01 17:01:31.107000+00:00 | 1 | 1 | None | None | 1 | None |
| 1 | ITeOtm7bhtdq0000 | transfer.ipynb | Transfer data | notebook | None | None | None | None | None | False | None | True | False | 2025-12-01 17:01:28.435000+00:00 | 1 | 1 | None | None | 1 | None |
******************
* module: bionty *
******************
Source
| uid | entity | organism | name | in_db | currently_used | description | url | md5 | source_website | version | is_locked | created_at | branch_id | space_id | created_by_id | run_id | dataframe_artifact_id | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | ||||||||||||||||||
| 33 | 5JnVODh4 | BioSample | all | ncbi | False | True | NCBI BioSample attributes | s3://bionty-assets/df_all__ncbi__2023-09__BioS... | None | https://www.ncbi.nlm.nih.gov/biosample/docs/at... | 2023-09 | False | 2025-12-01 17:01:24.364000+00:00 | 1 | 1 | 1 | None | None |
| 32 | MJRqduf9 | bionty.Ethnicity | human | hancestro | False | True | Human Ancestry Ontology | http://purl.obolibrary.org/obo/hancestro/relea... | None | https://github.com/EBISPOT/hancestro | 3.0 | False | 2025-12-01 17:01:24.364000+00:00 | 1 | 1 | 1 | None | None |
| 31 | 10va5JSt | bionty.DevelopmentalStage | mouse | mmusdv | False | True | Mouse Developmental Stages | https://github.com/obophenotype/developmental-... | None | https://github.com/obophenotype/developmental-... | 2024-05-28 | False | 2025-12-01 17:01:24.364000+00:00 | 1 | 1 | 1 | None | None |
| 30 | 1GbFkOdz | bionty.DevelopmentalStage | human | hsapdv | False | True | Human Developmental Stages | https://github.com/obophenotype/developmental-... | None | https://github.com/obophenotype/developmental-... | 2024-05-28 | False | 2025-12-01 17:01:24.364000+00:00 | 1 | 1 | 1 | None | None |
| 29 | 1atB0WnU | Drug | all | chebi | False | False | Chemical Entities of Biological Interest | s3://bionty-assets/df_all__chebi__2024-07-27__... | None | https://www.ebi.ac.uk/chebi/ | 2024-07-27 | False | 2025-12-01 17:01:24.364000+00:00 | 1 | 1 | 1 | None | None |
| 28 | ugaIoIlj | Drug | all | dron | False | True | Drug Ontology | http://purl.obolibrary.org/obo/dron/releases/2... | None | https://bioportal.bioontology.org/ontologies/DRON | 2024-08-05 | False | 2025-12-01 17:01:24.364000+00:00 | 1 | 1 | 1 | None | None |
| 27 | 3rm9aOzL | BFXPipeline | all | lamin | False | True | Bioinformatics Pipeline | s3://bionty-assets/df_all__lamin__1.0.0__BFXpi... | None | https://lamin.ai | 1.0.0 | False | 2025-12-01 17:01:24.364000+00:00 | 1 | 1 | 1 | None | None |
View lineage:
artifact.view_lineage()
Show code cell output
! calling anonymously, will miss private instances
The transferred dataset is linked to a special type of transform that stores the slug and uid of the source instance:
artifact.transform.description
Show code cell output
'Transfer from `laminlabs/lamindata`'
The transform key has the form f"__lamindb_transfer__/{source_instance.uid}":
artifact.transform.key
Show code cell output
'__lamindb_transfer__/4XIuR0tvaiXM'
The current notebook run is linked as the initiated_by_run of the “transfer run”:
artifact.run.initiated_by_run.transform
Show code cell output
Transform(uid='ITeOtm7bhtdq0000', version=None, is_latest=True, key='transfer.ipynb', description='Transfer data', type='notebook', hash=None, reference=None, reference_type=None, config=None, is_flow=False, flow=None, environment=None, branch_id=1, space_id=1, created_by_id=1, created_at=2025-12-01 17:01:28 UTC, is_locked=False)
Upon re-transferring a record, it will identify that the record already exists in the target database and simply map the record.
artifact = artifacts.filter(description__contains="Tabula Sapiens").first()
artifact.save()
Show code cell output
→ mapped: Artifact(uid='dPraor9rU1EofcFb6Wph')
Artifact(uid='dPraor9rU1EofcFb6Wph', version=None, is_latest=True, key='tabula_sapiens_lung.h5ad', description='Part of Tabula Sapiens, a benchmark, first-draft human cell atlas.', suffix='.h5ad', kind=None, otype=None, size=3899435772, hash='8mB1KK2wd51F6HQdvqipcQ', n_files=None, n_observations=None, params=None, branch_id=1, space_id=1, storage_id=2, run_id=2, schema_id=None, created_by_id=1, created_at=2023-07-14 19:00:30 UTC, is_locked=False)
If you also want to transfer annotations of the artifact, you can pass transfer="annotations" to save(). Just note that this might populate your target database with metadata that doesn’t match the conventions you want to enforce.
artifact = artifacts.filter(description__contains="Tabula Sapiens").first()
artifact.save(transfer="annotations")
Show code cell output
→ mapped: Artifact(uid='dPraor9rU1EofcFb6Wph'), Tissue(uid='7Tt4iEKc'), CellType(uid='5tiBvp96'), CellType(uid='7Crr32HI'), CellType(uid='6dzoXJ3Y'), CellType(uid='01NqvhnI'), CellType(uid='5NceZTYm'), CellType(uid='4PSMdO3I'), CellType(uid='3JO0EdVd'), CellType(uid='6rfrjhvo'), CellType(uid='37mWPv6o'), CellType(uid='5Z76sCep'), CellType(uid='2OWUH6Z1'), CellType(uid='5TU8SFt5'), CellType(uid='ryEtgi1y'), CellType(uid='1lMgAPE8'), CellType(uid='7m6Ruz32'), CellType(uid='42qbvc90'), CellType(uid='puGNwNrs'), CellType(uid='1T8bGe2I'), CellType(uid='6IC9NGJE'), CellType(uid='6ujMwy7s'), CellType(uid='3eecYgWR'), CellType(uid='zQ4dyjEs'), CellType(uid='7mNqzyFE'), CellType(uid='5A9EFjNB'), CellType(uid='3lsrLTv6'), CellType(uid='1HYtHpIc'), CellType(uid='6UmKFrzn'), CellType(uid='7eZArDpo'), CellType(uid='2KCFdGIk'), CellType(uid='1V5wVqK5'), CellType(uid='5i19XYug'), CellType(uid='2nPA0h4F'), CellType(uid='5Xi2OLvZ'), CellType(uid='3kaL3W1c'), ExperimentalFactor(uid='5YDCOg0V'), ExperimentalFactor(uid='7R1OhRJ7')
→ transferred: ULabel(uid='vfLXaHgD'), ULabel(uid='ZaVLDCZE'), ULabel(uid='gk6w8qC5'), ULabel(uid='tZCTk48f'), CellType(uid='4mZaXZQg'), CellType(uid='5rVn0X39'), CellType(uid='EWy46Sey'), CellType(uid='4yqLzwwm')
Artifact(uid='dPraor9rU1EofcFb6Wph', version=None, is_latest=True, key='tabula_sapiens_lung.h5ad', description='Part of Tabula Sapiens, a benchmark, first-draft human cell atlas.', suffix='.h5ad', kind=None, otype=None, size=3899435772, hash='8mB1KK2wd51F6HQdvqipcQ', n_files=None, n_observations=None, params=None, branch_id=1, space_id=1, storage_id=2, run_id=2, schema_id=None, created_by_id=1, created_at=2023-07-14 19:00:30 UTC, is_locked=False)
The artifact is now annotated.
artifact.describe()
Show code cell output
Artifact: tabula_sapiens_lung.h5ad (6Wph) | description: Part of Tabula Sapiens, a benchmark, first-draft human cell atlas. ├── uid: dPraor9rU1EofcFb6Wph run: 8Kxl2LV (__lamindb_transfer__/4XIuR0tvaiXM) │ hash: 8mB1KK2wd51F6HQdvqipcQ size: 3.6 GB │ branch: main space: all │ created_at: 2023-07-14 19:00:30 UTC created_by: anonymous ├── storage/path: s3://lamindata/tabula_sapiens_lung.h5ad └── Labels └── .ulabels ULabel TSP1, TSP2, TSP14 .tissues bionty.Tissue lung .cell_types bionty.CellType pulmonary alveolar type 1 cell, adventi… .experimental_factors bionty.ExperimentalFactor anoxya, stroke
Show code cell content
# test the last 3 cells here
assert artifact.transform.description == "Transfer from `laminlabs/lamindata`"
assert artifact.transform.key == "__lamindb_transfer__/4XIuR0tvaiXM"
assert artifact.transform.uid == "4XIuR0tvaiXM0000"
assert artifact.run.initiated_by_run.transform.description == "Transfer data"