elinkapi 0.4.8

OSTI E-Link 2 API package

ELINKAPI - A Python Interface for E-Link 2.0

Table of Contents

• ELINKAPI - A Python Interface for E-Link 2.0
  • Open Questions
  • Table of Contents
  • Introduction
  • Installation
    • Importing the Package from test.pypi.org
    • Importing the Package from Production PyPI
  • Examples
    • Using the API connection
    • Creating a New Record
    • Seeing Validation Errors on Exception
    • Processing Status of Submitted Records
    • View Revision History
    • Adding Media to Record
    • Removing Metadata Records
    • Removing Media from a Record
    • Compare Two Revision Histories
    • Searching and pagination
  • Method Documentation
    • Configuration
    • Records
      • Revisions
    • Media
  • Classes
    • Record
    • Query
    • Organization
    • Person
    • Identifier
    • Related Identifier
    • Geolocation
    • Media Info
    • Media File
    • Revision
    • Revision Comparison
  • Exceptions
    • UnauthorizedException
    • ForbiddenException
    • BadRequestException
    • NotFoundException
    • ConflictException
    • ServerException

Introduction

This module is setup to mimic the E-Link 2.0 API Endpoints (API documentation found here) and allows for you to quickly get up and running submitting Records using Python.

Installation

Importing the Package from test.pypi.org

1. Install the package, but don't grab the dependencies (pip will attempt to grab everything from the test server, which we do not want): pip install --index-url https://test.pypi.org/simple/ --no-deps elinkapi
2. Now install the other dependencies: pip install elinkapi
3. Or install them separately: pip install requests pydantic urllib3==1.26.6
4. Access the E-Link connector via from elinkapi import Elink and creating an instance for use with your API key: api = Elink(token="Your_API_Token")
5. API classes are accessible using from elinkapi import Record, etc.
6. Exception classes generated by the API are accessible using from elinkapi import exceptions then catching appropriate exceptions.BadRequestException and the like.

Importing the Package from Production PyPI

1. Install the package: pip install elinkapi
2. Access the E-Link connector via from elinkapi import Elink and creating an instance for use with your API key: api = Elink(token="Your_API_Token")
3. API classes are accessible using from elinkapi import Record, etc.
4. Exception classes generated by the API are accessible using from elinkapi import exceptions then catching appropriate exceptions.BadRequestException and the like.

Examples

Using the API connection

In order to use the python API library, it is generally required to create an instance of the "API object" to interact with. Following examples will start out with this, but more details will be provided in this section.

The API instance sets the target URL value (that is, the desired E-Link service) to connect to; by default this will be the production E-Link service. For testing or pre-release Review or Beta access, specify a target to the review E-Link service. Please note that each of these is an entirely separate service; they do not share records, user accounts, or other information. The Review E-Link is intended for testing your code, or for new yet-to-be-released features and interface functionality, and does not constitute release of STI information to the production OSTI.gov web site.

It is not required to maintain separate python library dependencies for these different environments; the library instance may connect to either service depending on how you instantiate it.

from elinkapi import Elink

api = Elink(target="TARGET_URL", token="__Your_API_Token__")

Again, if you do not specify the TARGET_URL above, the production E-Link will be utilized. Specify https://review.osti.gov/elink2api/ as the TARGET_URL above to access the Review E-Link environment. Adjust the token value to the appropriate API user token for the correct target environment prior to utilizing the functions.

If you wish to interactively change either value, there are convenience methods available:

# change the TARGET to review
api.set_target_url("https://review.osti.gov/elink2api/")
# but also don't forget to change your token!
api.set_api_token("MYNEWTOKENVALUE")
# you may also view these settings in python
api.target
'https://review.osti.gov/elink2api/'
api.token
'MYNEWTOKENVALUE'

Creating a New Record

Note: Ensure site_ownership_code is a value to which your user account token has sufficient access to create records.

from elinkapi import Elink, Record, exceptions

api = Elink(token="__Your_API_Token__")

# Record with minimal fields to save
my_record_json = {
        "title": "A Dissertation Title",
        "site_ownership_code": "AAAA",
        "product_type": "TD"
        }
# Convert json to Record object
my_record = Record(**my_record_json)

saved_record = None
try:
    saved_record = api.post_new_record(my_record, "save")
except exceptions.BadRequestException as ve:
    # ve.message = "Site Code AAAA is not valid."
    # ve.errors provides more details:
    # [{"status":"400", "detail":"Site Code AAAA is not valid.", "source":{"pointer":"site_ownership_code"}}]

Seeing Validation Errors on Exception

from elinkapi import Elink, Record, BadRequestException

# Record missing fields, will give 2 validation errors, one for 
# each missing field: title and product_type
my_invalid_record_json = {
    "site_ownership_code": "AAAA"
}

try:
    # The pydantic model will raise exceptions for the 2 missing 
    # fields - title and product_type
    my_record = Record(**my_invalid_record_json)
except Exception as e:
    print('Exception on Record creation')
    # pydantic will return "missing" required fields as below:
    # 2 validation errors for Record
    # product_type
    #    Field required [type=missing, input_value={'site_ownership_code': 'BBBB'}, input_type=dict]
    #    For further information visit https://errors.pydantic.dev/2.6/v/missing
    # title
    #    Field required [type=missing, input_value={'site_ownership_code': 'BBBB'}, input_type=dict]
    #    For further information visit https://errors.pydantic.dev/2.6/v/missing

my_invalid_record_json = {
    "title": "A Sample Title",
    "product_type": "TD",
    "site_ownership_code": "AAAA"
}

my_record = Record(**my_invalid_record_json)

saved_record = None
try:
    # The API will now return an error code on this call
    # because "AAAA" is not a valid site_ownership_code
    saved_record = api.post_new_record(my_record, "save")
except exceptions.BadRequestException as ve:
    # E-Link BadRequestException provides details of the API response:
    # ve.message = "Site Code AAAA is not valid."
    # ve.errors provides more details:
    # [{"status":"400", "detail":"Site Code AAAA is not valid.", "source":{"pointer":"site_ownership_code"}}]

Processing Status of Submitted Records

When records are submitted to the E-Link API (either via POST for a new record, or PUT to update a revision) and validation is successful, the E-Link release processing services immediately begin to process the record. This often will transition the workflow_status of the current revision through the initial "SO" through "SV" to a final "R" of a successful release.

During this processing, a record might stop at the "SV" state (if awaiting media processing, for example) or end up in "SF" (submitted, but failed validations) state. In these cases, or if additional information is desired for fully-released records, the auditing logs are available to query, showing the processing states and workers involved, or any issues in processing that might be preventing release of the revision.

Simply use the get_single_record API function call with a given OSTI ID:

from elinkapi import Elink

api = Elink(token = "__Your_API_Token__")

record = api.get_single_record(osti_id_value)

# "raw" audit logs
record.audit_logs
[AuditLog(messages=['Revision passed validation'], status='SUCCESS', type='VALIDATOR', audit_date=datetime.datetime(2024, 2, 13, 17, 0, 4, 76000, tzinfo=TzInfo(UTC)))]

# obtain audit log information in JSON
[l.model_dump_json() for l in record.audit_logs]

['{"messages":["Revision passed validation"], "status":"SUCCESS", "type": "VALIDATOR", "audit_date":"2024-02-13T17:00:04.076000Z"}']

Audit logs will typically include references from "VALIDATOR", "RELEASER", "DOI", and other microservices as applicable. Each audit event will detail the date-time performed, and a status of either "SUCCESS" or "FAIL". This will provide more insight into the processing of individual submitted records. Once a revision fully releases (workflow_status="R") it should become available on the OSTI.gov search service (in production API instances).

Note that most API responses, including records and many of their component child data (identifiers, etc.) are returned as pydantic data classes. If information in JSON is desired, use the model_dump_json() method available to such output classes.

View Revision History

from elinkapi import Elink

api = Elink(token="__Your_API_Token__")

osti_id = 99999999

revision_history = None
try:
    revision_history = api.get_all_revisions(osti_id)
except Exception as e:
    # Handle the exception as needed

most_recent_revision = revision_history[0]
oldest_revision = revision_history[-1]

Various components of record history are available in each revision summary, such as the date_valid_start and date_valid_end, workflow_status, etc. The dates indicate the time frame during which that revision was "current"; if date_valid_end is null, this indicates that particular revision is the current one.

Adding Media to Record

from elinkapi import Elink

api = Elink(token = '__Your_API_Token__')

osti_id = 9999999
path_to_my_media = "/home/path/to/media.pdf"

saved_media = None
try:
    saved_media = api.post_media(osti_id, path_to_my_media)
except Exception as e:
    # Handle the exception as needed

Removing Record

from elinkapi import Elink

api = Elink(token = "___Your-API-Token___")

try:
    response = api.delete_record(99999, "Resource no longer available")
except Exception as e:
    # Handle any exceptions

Removing Media from a Record

from elinkapi import Elink

api = Elink(token = "___Your-API-Token___")

osti_id = 9999999
media_id = 71
reason = "Uploaded the wrong file"

response = None
try:
    response = api.delete_single_media(osti_id, media_id, reason)
except Exception as e:
    # Handle the exception as needed

Compare Two Revision Histories

from elinkapi import Elink

api = Elink(token = "___Your-API-Token___")

osti_id = 2300069
revision_id_left = 1
revision_id_right = 2

response = None
try:
    response = elinkapi.compare_two_revisions(osti_id, revision_id_left, revision_id_right)
except Exception as e:
    # Handle the exception as needed

Searching and pagination

from elinkapi import Elink, Query

api  = Elink(token = "___Your-API-Token___")

query = api.query_records(title = "science", product_type = "JA")

# see number of results
print (f"Query matched {query.total_rows} records")

# paginate through ALL results using iterator
for record in query:
    print (f"OSTI ID: {record.osti_id} Title: {record.title}")

Searches are limited via keywords specified in the query_records method call. Search term fields and further information is available in the online API documentation.

Method Documentation

Configuration

The following methods may alter parameters on existing Elink instances to alter or set values.

from elinkapi import Elink

# you may set these directly or alter them later
# note target defaults to "https://www.osti.gov/elink2api/" for the E-Link API
# to access Review or Beta environment, you may specify the target:
api = Elink(token = 'TOKENVALUE', target="https://review.osti.gov/elink2api/")

# change them
api.set_api_token("NEWTOKEN")
# change to PRODUCTION E-Link API
api.set_target_url("https://www.osti.gov/elink2api/")

Method:

set_api_token(api_token)

Returns: None

Params:

• api_token - str: Unique to user API token that can be generated from your E-Link Account page

---

Method:

set_target_url(url="https://www.osti.gov/elink2api"):

Returns: None

Params:

• url - str: The url to which all other module methods will direct their requests (default: {"https://www.osti.gov/elink2api"})

---

Records

Method:

get_single_record(osti_id)

Returns: Record

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record

---

Method:

query_records(params)

Example:

api.query_records(title="science")

Returns: Query object

Params:

• params - dict: See here for the list of allowed query parameters.

---

Method:

reserve_doi(record)

Returns: Record

Params:

• record - Record: Metadata record that you wish to save to E-Link 2.0

---

Method:

post_new_record(record, state="save")

Returns: Record

Params:

• record - Record: Metadata record that you wish to send ("save" or "submit") to E-Link 2.0
• state - str: The desired submission state of the record ("save" or "submit") (default: {"save"})

---

Method:

update_record(osti_id, record, state="save")

Returns: Record

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• record - Record: Metadata record that you wish to make the new revision of OSTI ID
• state - str: The desired submission state of the record ("save" or "submit") (default: {"save"})

---

Revisions

Method:

get_revision_by_number(osti_id, revision_number)

Returns: Record

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• revision_number - int: The specific revision number to retrieve (original record is 1 and each revision increments upward by 1)

---

Method:

get_revision_by_date(osti_id, date)

Returns: Record

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• date - datetime: Date on which you wish to search for a revision of a Record

---

Method:

get_all_revisions(osti_id)

Returns: RevisionHistory

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record

---

Method:

compare_two_revisions(osti_id, left, right)

Returns: List[RevisionComparison]

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• left - int: The first revision number to retrieve and compare to the right
• right - int The second revision number to retrieve and compare to the left

---

Media

Method:

get_media(osti_id)

Returns: MediaInfo

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record

---

Method:

get_media_content(media_file_id)

Returns: Byte string of the media file content

Params:

• media_file_id - int: ID that uniquely identifies a media file associated with an E-Link 2.0 Record

---

Method:

post_media(osti_id, file_path, params=None, stream=None)

Returns: MediaInfo

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• file_path - str: Path to the media file that will be attached to the Record
• params - dict: "title" that can be associated with the media file "url" that points to media if not sending file (default: {None})
• stream - bool: Whether to stream the media file data, which has better performance for larger files (default: {False})

---

Method:

put_media(osti_id, media_id, file_path, params=None, stream=None)

Returns: MediaInfo

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• media_id - int: ID that uniquely identifies a media file associated with an E-Link 2.0 Record
• file_path - str: Path to the media file that will replace media_id Media
• params - dict: "title" that can be associated with the media file "url" that points to media if not sending file (default: {None})
• stream - bool: Whether to stream the media file data, which has better performance for larger files (default: {False})

---

Method:

delete_all_media(osti_id, reason)

Returns: True on success, False on failure

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• reason - str: reason for deleting all media

---

Method:

delete_single_media(osti_id, media_id, reason)

Returns: True on success, False on failure

Params:

• osti_id - int: ID that uniquely identifies an E-Link 2.0 Record
• media_id - int: ID that uniquely identifies a media file associated with an E-Link 2.0 Record
• reason - str: reason for deleting media

---

Classes

Each class is a pydantic model that validates the metadata's data types and enumerated values on instantiation of the class. Each may be imported directly:

from elinkapi import Record, Organization, Person, Query, Identifier, RelatedIdentifier, Geolocation, MediaInfo, MediaFile
from elinkapi import Revision, RevisionComparison

Record

Matches the Metadata model described in E-Link 2.0's API documentation

Query

Produced by API query searches, enables pagination and access to total count of rows matching the query. Query is iterable, and may use Python constructs to paginate all results as desired. Iterating over the Query returns metadata records individually.

See Searching and pagination for example and more information.

Provides:

• total_rows - int: Total count of records matching the query
• has_next() - boolean: True if there are more results to be fetched
• has_previous() - boolean: True if there is a previous page of results

Organization

Matches the Organizations model described in E-Link 2.0's API documentation

Person

Matches the Persons model described in E-Link 2.0's API documentation

Identifier

Matches the Identifiers model described in E-Link 2.0's API documentation

Related Identifier

Matches the Related Identifiers model described in E-Link 2.0's API documentation

Geolocation

Schema

Geolocation: {
    "type": str
    "label": str
    "points": List[Point]
}
Point: {
    "latitude": float
    "longitude": float
}

Example

{
    "type": "BOX",
    "label": "Utah FORGE",
    "points": [
        {
            "latitude": 38.5148,
            "longitude": -112.879748
        },
        {
            "latitude": 38.483935,
            "longitude": 112.916367
        }
    ]
}

Media Info

Schema

[
    {
        "media_id": int,
        "revision": int,
        "access_limitations": List[str],
        "osti_id": int,
        "status": str,
        "added_by": int,
        "document_page_count": int,
        "mime_type": str,
        "media_title": str,
        "media_location": str,
        "media_source": str,
        "date_added": datetime,
        "date_updated": datetime,
        "date_valid_start": datetime,
        "date_valid_end": datetime,
        "files": List[MediaFile]
    }
]

Example

[
    {
        "media_id": 233743,
        "revision": 3,
        "access_limitations": [],
        "osti_id": 99238,
        "status": "P",
        "added_by": 34582,
        "document_page_count": 23,
        "mime_type": "application/pdf",
        "media_title": "PDF of technical report content",
        "media_location": "L",
        "media_source": "MEDIA_API_UPLOAD",
        "date_added": "1992-03-08T11:23:44.123+00:00",
        "date_updated": "2009-11-05T08:33:12.231+00:00",
        "date_valid_start": "2021-02-13T16:32:23.234+00:00",
        "date_valid_end": "2021-02-15T12:32:11.332+00:00",
        "files": []
    }
]

Media File

Schema

{
    "media_file_id": int,
    "media_id": int,
    "revision": int,
    "status": str,
    "media_type": str,
    "url_type": str,
    "added_by_user_id": int,
    "file_size_bytes": int,
    "date_file_added": datetime,
    "date_file_updated": datetime"
}

Example

{
    "media_file_id": 12001019,
    "media_id": 1900094,
    "revision": 2,
    "status": "ADDED",
    "media_type": "O",
    "url_type": "L",
    "added_by_user_id": 112293,
    "file_size_bytes": 159921,
    "date_file_added": "2023-12-20T22:13:16.668+00:00",
    "date_file_updated": "2023-12-20T22:13:16.668+00:00"
}

Revision

Schema

{
    "date_valid_start": datetime,
    "date_valid_end": datetime,
    "osti_id": int,
    "revision": int,
    "workflow_status": str
}

Example

{
    "date_valid_start": "2022-12-04T13:22:45.092+00:00",
    "date_valid_end": "2023-12-04T13:22:45.092+00:00",
    "osti_id": 2302081,
    "revision": 2,
    "workflow_status": "R"
}

Revision Comparison

Schema

[
    {
        "date_valid_start": datetime,
        "date_valid_end": datetime,
        "osti_id": int,
        "revision": int,
        "workflow_status": str
    }
]

Example

[
    {
        "pointer": "/edit_reason",
        "left": "API record creation",
        "right": "API metadata Update"
    },
    {
        "pointer": "/description",
        "left": "A custom description. Search on 'Allo-ballo holla olah'.",
        "right": "A NEW custom description. Search on 'Allo-ballo holla olah'."
    }
]

Exceptions

Various exceptions are raised via API calls, and may be imported and caught in the code. Using

from elinkapi import exceptions

will provide access to the various exception types for handling.

UnauthorizedException

Generally raised when no API token value is provided when accessing E-Link.

ForbiddenException

Raised when attempting to query records, post new content to a site, or create/update records to which the API token has no permission.

BadRequestException

Raised when provided query parameters or values are not valid or not understood, or if validation errors occurred during submission of metadata. Additional details are available via the errors list, each element containing the following information about the various validation issues:

• detail: an error message indicating the issue
• source: contains a "pointer" to the JSON tag element in error

Example:

[{
  "detail":"Site Code BBBB is not valid.",
  "source":{
    "pointer":"site_ownership_code"
  }}]

NotFoundException

Raised when OSTI ID or requested resource is not on file.

ConflictException

Raised when attempting to attach duplicate media or URL to a given OSTI ID metadata.

ServerException

Raised if E-Link back end services or databases have encountered an unrecoverable error during processing.