Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Uploading Data to EarthScope - Tutorial

In this tutorial, we demonstrate programmatic interaction with EarthScope’s Dropoff system for data submission.

Setup

Create an EarthScopeClient instance and optionally declare a callback function to receive progress updates.

from earthscope_sdk import AsyncEarthScopeClient
from earthscope_sdk.client.dropoff.models import UploadStatus

es = AsyncEarthScopeClient()


def progress_callback(status: UploadStatus):
    print(status)

Upload an Object

Use the put_object() or put_objects() (plural) methods to upload objects to EarthScope’s Dropoff S3 Bucket.

When using put_object, you must provide:

  • body: the content to upload to S3. This can be any of the following:

    • a str or Path instance is interpreted as a path to a local file

    • a bytes instance is an in-memory buffer

    • a file-like object (any object with a .read() method) will be read in a streaming fashion

    • an iterator or async iterator is expected to yield bytes and will be read in a streaming fashion

  • key: the S3 object key where the object will be uploaded

  • category: the dropoff category. This dropoff system is used for many different data submission categories.

  • progress_cb (optional): a callback function to receive updates throughout the upload process

In the following example, we upload a local miniSEED file.

result = await es.dropoff.put_object(
    "/path/to/data/SAML.IU.2025.303",
    key="demo/SAML.IU.2025.303",
    category="miniseed",
    progress_cb=progress_callback,
)
print(f"Uploaded {result.key} ({result.size} bytes)")
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=0, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=10485760, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=20971520, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=31457280, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=41409536, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=9952256, bytes_buffered=41409536, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=20438016, bytes_buffered=41409536, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=30923776, bytes_buffered=41409536, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=41409536, bytes_buffered=41409536, total_bytes=41409536, complete=True)
Uploaded demo/SAML.IU.2025.303 (41409536 bytes)

Browsing Submitted Data

You can use list_objects(), get_object_history(), and get_summary() to explore the data you’ve submitted to EarthScope and check on the status of each submission.

The results of list_objects() and get_object_history() are paginated to handle large result sets.

Listing Objects

To browse through your dropoff space, use the list_objects() method. Optionally, provide a prefix= keyword argument to drill down into nested prefixes.

page = await es.dropoff.list_objects(prefix="demo/")
page.items[0]
DropoffObject(path='demo/SAML.IU.2025.303', received_at=datetime.datetime(2025, 12, 31, 21, 46, 31, tzinfo=TzInfo(0)), status='VALIDATING', status_message=None, size=41409536, sha256='c9a668471eda59068de73e3bcf46febd3fa05be025cae19182d1c0846b83f445')

Object History

When the same S3 key is used multiple times (i.e. to replace an errant submission), you can retrieve the history of each submission using get_object_history()

page = await es.dropoff.get_object_history(key="demo/SAML.IU.2025.303")
page.items
[DropoffObject(path='demo/SAML.IU.2025.303', received_at=datetime.datetime(2025, 12, 31, 21, 46, 31, tzinfo=TzInfo(0)), status='VALIDATING', status_message=None, size=41409536, sha256='c9a668471eda59068de73e3bcf46febd3fa05be025cae19182d1c0846b83f445'), DropoffObject(path='demo/SAML.IU.2025.303', received_at=datetime.datetime(2025, 12, 31, 21, 44, 37, tzinfo=TzInfo(0)), status='AUTHORIZED', status_message=None, size=41409536, sha256='c9a668471eda59068de73e3bcf46febd3fa05be025cae19182d1c0846b83f445'), DropoffObject(path='demo/SAML.IU.2025.303', received_at=datetime.datetime(2025, 12, 31, 21, 44, 19, tzinfo=TzInfo(0)), status='AUTHORIZED', status_message=None, size=41409536, sha256='c9a668471eda59068de73e3bcf46febd3fa05be025cae19182d1c0846b83f445'), DropoffObject(path='demo/SAML.IU.2025.303', received_at=datetime.datetime(2025, 12, 31, 21, 43, 28, tzinfo=TzInfo(0)), status='AUTHORIZED', status_message=None, size=41409536, sha256='c9a668471eda59068de73e3bcf46febd3fa05be025cae19182d1c0846b83f445'), DropoffObject(path='demo/SAML.IU.2025.303', received_at=datetime.datetime(2025, 12, 31, 21, 37, 45, tzinfo=TzInfo(0)), status='ACCEPTED', status_message=None, size=41409536, sha256='c9a668471eda59068de73e3bcf46febd3fa05be025cae19182d1c0846b83f445'), DropoffObject(path='demo/SAML.IU.2025.303', received_at=datetime.datetime(2025, 12, 31, 21, 37, 13, tzinfo=TzInfo(0)), status='ACCEPTED', status_message=None, size=41409536, sha256='c9a668471eda59068de73e3bcf46febd3fa05be025cae19182d1c0846b83f445'), DropoffObject(path='demo/SAML.IU.2025.303', received_at=datetime.datetime(2025, 12, 31, 21, 35, 31, tzinfo=TzInfo(0)), status='ACCEPTED', status_message=None, size=41409536, sha256='c9a668471eda59068de73e3bcf46febd3fa05be025cae19182d1c0846b83f445')]

Getting a Summary

Use get_summary() to retrieve a count of objects grouped by processing status for a given prefix. This gives you a quick health check of a batch submission without needing to page through individual file listings.

Optionally filter by submission time using submitted_after and submitted_before.

summary = await es.dropoff.get_summary(prefix="data/")
for item in summary.status_counts:
    print(f"{item.status}: {item.count}")
ACCEPTED: 102
FAILED: 3
VALIDATING: 42

Bulk Submission

When you need to upload more than one object, the put_objects() (plural) method should be used to more effectively saturate your upload bandwidth.

Instead of providing a single body and key, the objects argument is used to provide a list of (body, key) tuples.

  • the body of each object can still be any of the types supported by put_object() (singular)

  • objects may be an iterable of tuples which will be consumed in a streaming fashion

results = await es.dropoff.put_objects(
    objects=[
        (
            "/path/to/data/SAML.IU.2025.303#2",
            "demo/SAML.IU.2025.303",
        ),
        (
            "/path/to/data/BILL.IU.2025.303#2",
            "demo/BILL.IU.2025.303",
        ),
        # ... more (body, key) tuples ...
    ],
    category="miniseed",
    progress_cb=progress_callback,
)

for result in results:
    print(f"Uploaded {result.key} ({result.size} bytes)")
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=0, total_bytes=41409536, complete=False)
UploadStatus(key='demo/BILL.IU.2025.303', bytes_done=0, bytes_buffered=0, total_bytes=17494016, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=10485760, total_bytes=41409536, complete=False)
UploadStatus(key='demo/BILL.IU.2025.303', bytes_done=0, bytes_buffered=10485760, total_bytes=17494016, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=20971520, total_bytes=41409536, complete=False)
UploadStatus(key='demo/BILL.IU.2025.303', bytes_done=0, bytes_buffered=17494016, total_bytes=17494016, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=31457280, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=0, bytes_buffered=41409536, total_bytes=41409536, complete=False)
UploadStatus(key='demo/BILL.IU.2025.303', bytes_done=7008256, bytes_buffered=17494016, total_bytes=17494016, complete=False)
UploadStatus(key='demo/BILL.IU.2025.303', bytes_done=17494016, bytes_buffered=17494016, total_bytes=17494016, complete=True)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=10485760, bytes_buffered=41409536, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=20971520, bytes_buffered=41409536, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=30923776, bytes_buffered=41409536, total_bytes=41409536, complete=False)
UploadStatus(key='demo/SAML.IU.2025.303', bytes_done=41409536, bytes_buffered=41409536, total_bytes=41409536, complete=True)
Uploaded demo/SAML.IU.2025.303 (41409536 bytes)
Uploaded demo/BILL.IU.2025.303 (17494016 bytes)

Advanced Use Cases

Streaming Uploads

Our SDK will do its best to stream bodies to S3 rather than materialize them in memory in their entirety.

For example, in the following cell, we perform an HTTP request and upload its response to EarthScope’s Dropoff system. Instead of waiting for the response to download completely before we start the upload, the HTTP response is streamed into S3 without ever keeping the entire body in memory at any given time (unless the body is small).

import requests

result = await es.dropoff.put_object(
    body=requests.get(
        "https://service.earthscope.org/fdsnws/dataselect/1/query?net=IU&sta=ANMO&starttime=2025-11-15T00:00:00&endtime=2025-11-16T00:00:00&nodata=404",
        stream=True,
    ),
    key="demo/ANMO.IU.2025.2025.319.mseed",
    category="miniseed",
    progress_cb=progress_callback,
)
print(f"Uploaded {result.key} ({result.size} bytes)")
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=10485760, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=20971520, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=31457280, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=41943040, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=52428800, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=62914560, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=73400320, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=83886080, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=94371840, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=104857600, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=0, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=10485760, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=20971520, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=31457280, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=31682560, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=42168320, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=52654080, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=63139840, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=73625600, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=84111360, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=94597120, bytes_buffered=105082880, total_bytes=None, complete=False)
UploadStatus(key='demo/ANMO.IU.2025.2025.319.mseed', bytes_done=105082880, bytes_buffered=105082880, total_bytes=None, complete=True)
Uploaded demo/ANMO.IU.2025.2025.319.mseed (105082880 bytes)

Tuning Upload Behavior

The put_object and put_objects methods use S3 multipart uploads to facilitate efficient and resumable uploads. Multipart uploads break an object into multiple smaller “parts” that can upload independently in order to checkpoint upload progress and also parallelize uploads.

Depending on your upload bandwidth and connection reliability, it may be beneficial to tune the following parameters to better suit your situation.

  • object_concurrency: the total number of objects to upload at once

  • part_concurrency: the total number of object parts to upload at once

  • part_size: the size (in bytes) of each part to upload. the final part’s size may be less than part_size

For example, on a slow or intermittent internet connection, the following settings may help get the whole body to EarthScope’s S3 bucket by not spreading your upload bandwidth across multiple parts or objects at once. If the upload is interrupted, it can be resumed without needing to re-upload any parts that have already successfully been uploaded.

await es.dropoff.put_object(
    body=...,
    key=...,
    category=...,
    object_concurrency=1,  # only upload one object at a time
    part_concurrency=1,  # only upload one part at a time
    part_size=5 * 1024**2,  # 5 MB parts
)

Conversely, a high speed internet connection may benefit from more parallelism and/or larger part sizes.

await es.dropoff.put_object(
    body=...,
    key=...,
    category=...,
    object_concurrency=3,  # upload three objects at a time
    part_concurrency=12,  # upload twelve parts at a time (across all objects)
    part_size=16 * 1024**2,  # 16 MB parts
)
Data Contribution
Dropoff System Overview
How-To Guides
Direct S3 Access