EarthScope SDK Settings - EarthScope SDK

EarthScope SDK Settings consist of configuration & tokens.

Configuration is used to control behavior of the SDK.
Tokens are used to authenticate and authorize requests made by this SDK.

This document details the settings available, and a specification for settings fields, loading, and persistence on the user’s machine.

Note: if you don’t need to make any configuration changes (the most common use case!) then you don’t need to do anything and this page can be ignored.

Settings Fields¶

Configuration¶

Field	Environment Variable	Description
`profile_name`	`ES_PROFILE`	a named configuration profile
`dropoff.*`	`ES_DROPOFF__*`	Data dropoff configuration
`http.*`	`ES_HTTP__*`	HTTP client configuration
`oauth2.*`	`ES_OAUTH2__*`	oauth2 flow configuration
`query_plan.*`	`ES_QUERY_PLAN__*`	query plan configuration
`resources.*`	`ES_RESOURCES__*`	EarthScope resource references

Oauth2 Settings¶

Field	Environment Variable	Description
`oauth2.audience`	`ES_OAUTH2__AUDIENCE`	oauth2 audience targeted
`oauth2.client_id`	`ES_OAUTH2__CLIENT_ID`	oauth2 Client ID used when requesting tokens
`oauth2.client_secret`	`ES_OAUTH2__CLIENT_SECRET`	oauth2 Client Secret used for machine-to-machine auth
`oauth2.domain`	`ES_OAUTH2__DOMAIN`	oauth2 domain for Identity Provider
`oauth2.scope`	`ES_OAUTH2__SCOPE`	oauth2 scope requested during token retrieval
`oauth2.access_token`	`ES_OAUTH2__ACCESS_TOKEN`	oauth2 access token to use to authenticate with the API
`oauth2.refresh_token`	`ES_OAUTH2__REFRESH_TOKEN`	oauth2 refresh token used to retrieve new access tokens
`oauth2.id_token`	`ES_OAUTH2__ID_TOKEN`	OIDC token
`oauth2.allowed_hosts`	`ES_OAUTH2__ALLOWED_HOSTS`	Your access token is automatically sent when requests are made to these hosts. Supports glob-style strings with `?` and `*` characters.
`oauth2.retry.*`	`ES_OAUTH2__RETRY__*`	oauth2 retry configuration

Dropoff Settings¶

The EarthScope SDK’s S3 upload behavior can be modified to better suit your upload bandwidth.

Field	Environment Variable	Default	Description
`dropoff.bucket`	`ES_DROPOFF__BUCKET`		The S3 bucket to upload to.
`dropoff.category`	`ES_DROPOFF__CATEGORY`		A Dropoff category may be configured via settings to avoid specifying the category on each call to `put_object()`.
`dropoff.part_size`	`ES_DROPOFF__PART_SIZE`	10 MB	Size (in bytes) of each multipart upload part sent to S3. Larger parts mean fewer upload requests but higher memory use.
`dropoff.part_concurrency`	`ES_DROPOFF__PART_CONCURRENCY`	8	Maximum number of parts that will be uploaded in parallel (across multiple objects).
`dropoff.object_concurrency`	`ES_DROPOFF__OBJECT_CONCURRENCY`	3	Maximum number of objects that will be uploaded in parallel.
`dropoff.retry.*`	`ES_DROPOFF__RETRY__*`		S3 part upload retry configuration

HTTP Client Settings¶

Configuration for the HTTPx client & connection pool.

Field	Environment Variable	Description
`http.extra_headers`	`ES_HTTP__EXTRA_HEADERS`	Additional arbitrary headers to inject into requests
`http.user_agent`	`ES_HTTP__USER_AGENT`	The user agent injected into HTTP requests. This value supersedes a `user-agent` header passed via `http.extra_headers`.
`http.timeout_connect`	`ES_HTTP__TIMEOUT_CONNECT`	HTTP connect timeout (seconds)
`http.timeout_read`	`ES_HTTP__TIMEOUT_READ`	HTTP read/write timeout (seconds)
`http.keepalive_expiry`	`ES_HTTP__KEEPALIVE_EXPIRY`	Time limit on idle keep-alive connections (seconds)
`http.max_connections`	`ES_HTTP__MAX_CONNECTIONS`	The maximum number of concurrent connections that may be established
`http.max_keepalive_connections`	`ES_HTTP__MAX_KEEPALIVE_CONNECTIONS`	Allow the connection pool to maintain keep-alive connections below this point. Should be less than or equal to `max_connections`.
`http.rate_limit.*`	`ES_HTTP__RATE_LIMIT__*`	HTTP request rate limit configuration.
`http.retry.*`	`ES_HTTP__RETRY__*`	HTTP request retry configuration.

Query Plan Settings¶

Field	Environment Variable	Description
`query_plan.memory_limit_bytes`	`ES_QUERY_PLAN__MEMORY_LIMIT_BYTES`	Memory limit for query plan execution in bytes. If not set, no limit is enforced. Can be overridden per query.
`query_plan.timeout_seconds`	`ES_QUERY_PLAN__TIMEOUT_SECONDS`	Timeout for query plan execution in seconds. If not set, no timeout is enforced. Can be overridden per query.

Rate Limit Settings¶

Field	Environment Variable	Description
`max_concurrent`	`*__MAX_CONCURRENT`	Maximum number of concurrent requests
`max_per_second`	`*__MAX_PER_SECOND`	Maximum number of requests per second

Retry Settings¶

Field	Environment Variable	Description
`attempts`	`*__ATTEMPTS`	Maximum total number of attempts to perform when retryable errors occur.
`timeout`	`*__TIMEOUT`	Maximum total time for all retries when retryable errors occur.
`wait_initial`	`*__WAIT_INITIAL`	Minimum backoff before the first retry.
`wait_max`	`*__WAIT_MAX`	Maximum backoff time between retries at any time.
`wait_jitter`	`*__WAIT_JITTER`	Maximum jitter that is added to retry back-off delays (the actual jitter added is a random number between 0 and wait_jitter)
`wait_exp_base`	`*__WAIT_EXP_BASE`	The exponential base used to compute the retry backoff.

Resource References¶

References to EarthScope resources.

Field	Environment Variable	Description
`resources.api_url`	`ES_RESOURCES__API_URL`	the URL for EarthScope’s API

Settings Loading¶

SDK Settings are provided via the following methods (in order of precedence):

initialization arguments (e.g. via class constructors)
environment variables
dotenv file (.env) variables
user’s home directory settings files
1. ~/.earthscope/config.toml (for configuration)
2. ~/.earthscope/<profile-name>/tokens.json (for tokens)
legacy EarthScope CLI v0 credentials
bootstrapping environment: the environment variable ES_BOOTSTRAP_SETTINGS may contain a JSON string with any settings in the structure detailed above
default settings

SDK configuration is managed by the SdkSettings class, and calling the constructor performs this settings loading chain.

from earthscope_sdk.config.settings import SdkSettings

settings = SdkSettings()  # loads settings via loading chain

Initialization Arguments¶

Arguments to the SdkSettings class constructor take the highest precedence.

For example, the following will configure a custom oauth2 scope to request from the Identity Provider.

from earthscope_sdk.config.settings import SdkSettings

settings = SdkSettings(
  oauth2={"scope": "custom-scope"}
)

Environment Variables¶

Environment variables are a convenient way of configuring the SDK. The various settings are available with an ES_ prefix, as detailed in the tables above.

For example, the following will configure a custom oauth2 scope to request from the Identity Provider.

export ES_OAUTH2__SCOPE="custom-scope"

Settings Files¶

EarthScope SDK settings files live in the user’s home directory in a “hidden” directory called .earthscope.

Mac OS: ~/.earthscope/
Unix: ~/.earthscope/
Windows: C:\Users\<user>\.earthscope\

Configuration File¶

This file is completely optional.

Location: ~/.earthscope/config.toml

The EarthScope SDK can manage tokens for multiple named “profiles”. Profiles are used to manage multiple sets of tokens for interacting with different APIs, or the same API with different tokens.

You use config.toml to enumerate different profiles, and (optionally) unique configuration for each profile.

# ~/.earthscope/config.toml

# (optional) the `default` block contains fallback configuration for _all_ profiles
[default]
oauth2.scope = "default-custom-scope"
# ...etc (all configuration fields are available)

# (optional) `profile.<name>` block(s) define named configuration profile(s)
[profile.foo]
oauth2.scope = "profile-specific-custom-scope"
# ...etc (all configuration fields are available)

Tokens File(s)¶

Tokens for individual profiles are stored in their own JSON files:

Location: ~/.earthscope/<profile_name>/tokens.json

The SDK automatically manages these tokens.json files for you. You never need to manually create or edit these files.

{
  "access_token": "<access-token>", // oauth2 Access Token
  "id_token": "<id-token>", // (optional) oauth2 ID Token
  "refresh_token": "<refresh-token>" // (optional) oauth2 Refresh Token
}

Legacy EarthScope CLI v0 Credentials¶

The first version of the EarthScope CLI (v0) persisted tokens in a different location and with a different structure. To make upgrading from v0 to v1+ easier, the SDK will look for tokens in the legacy location.

Bootstrapping Environment Settings¶

Compute environments such as GeoLab may inject bootstrapping settings via the special ES_BOOTSTRAP_SETTINGS environment variable.

export ES_BOOTSTRAP_SETTINGS='{"oauth2": {"scope": "bootstrap-scope"}}'

Settings provided in this manner are superseded by all settings providers detailed above. This allows the compute environment (e.g. GeoLab) to inject configuration that may be overridden by users.

Default Settings¶

The SDK itself provides default values for all configuration fields. These default values are used as a last resort for any field that does not receive a higher-precedence value through the settings loading chain.

These defaults work out of the box for the primary use case of interacting with https://api.earthscope.org.