hydrotools.nwm-client

v8.0.0

Retrieve National Water Model data from various sources. For more information about how to use this package see README

Latest version published 18 days ago

License: “Software code created by U.S…

PyPI

GitHub

Package Health Score

72 / 100

security
No known security issues
popularity
Limited
maintenance
Healthy
community
Sustainable

Popularity

Limited

GitHub Stars: 54
Forks: 14
Contributors: 6

Direct Usage Popularity

Based on project statistics from the GitHub repository for the PyPI package hydrotools.nwm-client, we found that it has been starred 54 times.

Security

No known security issues

Security and license risk for latest version

Release Date: Dec 3, 2024
Direct Vulnerabilities: 0
C
0
H
0
M
0
L
Indirect Vulnerabilities: 0
C
0
H
0
M
0
L
License Risk: 0
H
0
M
0
L

All security vulnerabilities belong to production dependencies of direct and indirect packages.

License

“Software code created by U.S. Government employees is not subject to copyright in the United States (17 U.S.C. §105). The United States/Department of Commerce reserve all rights to seek and obtain copyright protection in countries other than the United States for Software authored in its entirety by the Department of Commerce. To this end, the Department of Commerce hereby grants to Recipient a royalty-free, nonexclusive license to use, copy, and create derivative works of the Software outside of the United States.”

Security Policy

Yes

You can connect your project's repository to Snyk to stay up to date on security alerts and receive automatic fix pull requests.

Keep your project free of vulnerabilities with Snyk

Maintenance

Healthy

Commit Frequency

Open Issues: 6
Open PR: 0
Last Release: 18 days ago
Last Commit: 18 days ago

Further analysis of the maintenance status of hydrotools.nwm-client based on released PyPI versions cadence, the repository activity, and other data points determined that its maintenance is Healthy.

We found that hydrotools.nwm-client demonstrates a positive version release cadence with at least one new version released in the past 3 months.

As a healthy sign for on-going project maintenance, we found that the GitHub repository had at least 1 pull request or issue interacted with by the community.

Community

Sustainable

Readme: Yes
Contributing.md: Yes
Code of Conduct: No
Contributors: 6
Funding: No

This project has seen only 10 or less contributors.

We found a way for you to contribute to the project! Looks like hydrotools.nwm-client is missing a Code of Conduct.

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Python Versions Compatibility: >=3.8

Age: 3 years
Latest Release: 18 days ago
Dependencies: 15 Direct
Versions: 5
Maintainers: 0
Wheels: OS Independent

Readme

OWPHydroTools :: NWM Client

This subpackage implements various interfaces to retrieve National Water Model (NWM) data from various sources including Google Cloud Platform, NOMADS, local directories, or a generic web server directory listing. The primary use for this tool is to populate pandas.Dataframe objects with NWM streamflow data. See the NWM Client Documentation for a complete list and description of the currently available methods. To report bugs or request new features, submit an issue through the OWPHydroTools Issue Tracker on GitHub.

Installation

In accordance with the python community, we support and advise the usage of virtual environments in any workflow using python. In the following installation guide, we use python's built-in venv module to create a virtual environment in which the tool will be installed. Note this is just personal preference, any python virtual environment manager should work just fine (conda, pipenv, etc. ).

# Create and activate python environment, requires python &gt;= 3.8
$ python3 -m venv env
$ source env/bin/activate
$ python3 -m pip install --upgrade pip wheel

# Install nwm_client
$ python3 -m pip install hydrotools.nwm_client

Usage

The following example demonstrates how one might use hydrotools.nwm_client to retrieve NWM streamflow forecasts.

Code

View compatible configurations

# Import the NWM Client
from hydrotools.nwm_client.NWMFileClient import NWMFileClient

# Instantiate model data client
model_data_client = NWMFileClient()

# Print compatible model configurations
#  Note that not all data sources contain the full range of available 
#  National Water Model data. This client defaults to Google Cloud Platform
#  Which has the largest amount of *operational* forecast data.
#  Also note that not all configurations are available for the entire
#  archive of NWM operational forecast data. For example, the configurations 
#  for Alaska only became available after August 2023.
print(model_data_client.catalog.configurations)

Example output

['analysis_assim', 'analysis_assim_alaska', 'analysis_assim_alaska_no_da', 'analysis_assim_extend', 'analysis_assim_extend_no_da', 'analysis_assim_extend_alaska', 'analysis_assim_extend_alaska_no_da', 'analysis_assim_hawaii', 'analysis_assim_hawaii_no_da', 'analysis_assim_no_da', 'analysis_assim_puertorico', 'analysis_assim_puertorico_no_da', 'analysis_assim_long', 'analysis_assim_long_no_da', 'long_range_mem1', 'long_range_mem2', 'long_range_mem3', 'long_range_mem4', 'medium_range_alaska_mem1', 'medium_range_alaska_mem2', 'medium_range_alaska_mem3', 'medium_range_alaska_mem4', 'medium_range_alaska_mem5', 'medium_range_alaska_mem6', 'medium_range_alaska_no_da', 'medium_range_mem1', 'medium_range_mem2', 'medium_range_mem3', 'medium_range_mem4', 'medium_range_mem5', 'medium_range_mem6', 'medium_range_mem7', 'medium_range_no_da', 'short_range', 'short_range_alaska', 'short_range_hawaii', 'short_range_hawaii_no_da', 'short_range_puertorico', 'short_range_puertorico_no_da']

Retrieving data from google cloud

# Import the NWM Client
from hydrotools.nwm_client.NWMFileClient import NWMFileClient

# Instantiate model data client
#  By default, NWM values are in SI units
#  If you prefer US standard units, nwm_client can return
#  values in US standard units by setting the unit_system parameter 
#  to MeasurementUnitSystem.US
# 
# from hydrotools.nwm_client.NWMClientDefaults import MeasurementUnitSystem
# model_data_client = NWMFileClient(unit_system=MeasurementUnitSystem.US)
model_data_client = NWMFileClient()

# Retrieve forecast data
forecast_data = model_data_client.get(
    configurations = ["short_range"],
    reference_times = ["20210101T01Z"],
    nwm_feature_ids = [724696]
    )

# Look at the data
print(forecast_data.head())

Example output

       reference_time  nwm_feature_id          value_time      value measurement_unit variable_name configuration usgs_site_code
0 2021-01-01 01:00:00          724696 2021-01-01 02:00:00  56.340000           m3 s-1    streamflow   short_range       01013500
1 2021-01-01 01:00:00          724696 2021-01-01 17:00:00  56.090000           m3 s-1    streamflow   short_range       01013500
2 2021-01-01 01:00:00          724696 2021-01-01 16:00:00  56.119999           m3 s-1    streamflow   short_range       01013500
3 2021-01-01 01:00:00          724696 2021-01-01 15:00:00  56.149998           m3 s-1    streamflow   short_range       01013500
4 2021-01-01 01:00:00          724696 2021-01-01 14:00:00  56.180000           m3 s-1    streamflow   short_range       01013500

Retrieving data from Azure Blob Storage

# Import the NWM Client
from hydrotools.nwm_client.NWMFileClient import NWMFileClient
from hydrotools.nwm_client.AzureFileCatalog import AzureFileCatalog
import pandas as pd

# Instantiate model data client
catalog = AzureFileCatalog()
model_data_client = NWMFileClient(catalog=catalog)

# Set reference time
yesterday = pd.Timestamp.utcnow() - pd.Timedelta("1D")

# Retrieve forecast data
forecast_data = model_data_client.get(
    configurations = ["short_range"],
    reference_times = [yesterday],
    nwm_feature_ids = [724696]
    )

# Look at the data
print(forecast_data.head())

Example output

       reference_time  nwm_feature_id          value_time      value measurement_unit variable_name configuration usgs_site_code
0 2022-08-07 18:00:00          724696 2022-08-07 19:00:00  20.369999           m3 s-1    streamflow   short_range       01013500
1 2022-08-07 18:00:00          724696 2022-08-08 10:00:00  24.439999           m3 s-1    streamflow   short_range       01013500
2 2022-08-07 18:00:00          724696 2022-08-08 09:00:00  24.469999           m3 s-1    streamflow   short_range       01013500
3 2022-08-07 18:00:00          724696 2022-08-08 08:00:00  24.490000           m3 s-1    streamflow   short_range       01013500
4 2022-08-07 18:00:00          724696 2022-08-08 07:00:00  24.510000           m3 s-1    streamflow   short_range       01013500

Retrieving data from Nomads

# Import the NWM Client
from hydrotools.nwm_client.NWMFileClient import NWMFileClient
from hydrotools.nwm_client.HTTPFileCatalog import HTTPFileCatalog
import pandas as pd

# Instantiate model data client
catalog = HTTPFileCatalog("https://nomads.ncep.noaa.gov/pub/data/nccf/com/nwm/prod/")
model_data_client = NWMFileClient(catalog=catalog)

# Set reference time
yesterday = pd.Timestamp.utcnow() - pd.Timedelta("1D")

# Retrieve forecast data
forecast_data = model_data_client.get(
    configurations = ["short_range"],
    reference_times = [yesterday],
    nwm_feature_ids = [724696]
    )

# Look at the data
print(forecast_data.head())

Example output

       reference_time  nwm_feature_id          value_time      value measurement_unit variable_name configuration usgs_site_code
0 2022-08-07 18:00:00          724696 2022-08-07 19:00:00  20.369999           m3 s-1    streamflow   short_range       01013500
1 2022-08-07 18:00:00          724696 2022-08-08 10:00:00  24.439999           m3 s-1    streamflow   short_range       01013500
2 2022-08-07 18:00:00          724696 2022-08-08 09:00:00  24.469999           m3 s-1    streamflow   short_range       01013500
3 2022-08-07 18:00:00          724696 2022-08-08 08:00:00  24.490000           m3 s-1    streamflow   short_range       01013500
4 2022-08-07 18:00:00          724696 2022-08-08 07:00:00  24.510000           m3 s-1    streamflow   short_range       01013500

Retrieving data from a private file server

# Import the NWM Client
from hydrotools.nwm_client.NWMFileClient import NWMFileClient
from hydrotools.nwm_client.HTTPFileCatalog import HTTPFileCatalog
from hydrotools.nwm_client.NWMClientDefaults import MeasurementUnitSystem
import ssl

# Create ssl context
context = ssl.create_default_context(cafile="/path/to/my/ca-bundle.crt")

# Instantiate model data client
catalog = HTTPFileCatalog(
    "https://path-to-my-private-server.com/nwm/2.2/", 
    ssl_context=context
    )
model_data_client = NWMFileClient(
    catalog=catalog,
    unit_system=MeasurementUnitSystem.US,
    ssl_context=context
)

# Retrieve forecast data
forecast_data = model_data_client.get(
    configurations = ["short_range"],
    reference_times = ["2022-06-01T13"],
    nwm_feature_ids = [724696]
    )

# Look at the data
print(forecast_data.head())

Example output

       reference_time  nwm_feature_id          value_time        value measurement_unit variable_name configuration usgs_site_code
0 2022-06-01 13:00:00          724696 2022-06-01 14:00:00  3586.910645           ft^3/s    streamflow   short_range       01013500
1 2022-06-01 13:00:00          724696 2022-06-02 05:00:00  2167.260986           ft^3/s    streamflow   short_range       01013500
2 2022-06-01 13:00:00          724696 2022-06-02 04:00:00  2168.673584           ft^3/s    streamflow   short_range       01013500
3 2022-06-01 13:00:00          724696 2022-06-02 03:00:00  2172.558350           ft^3/s    streamflow   short_range       01013500
4 2022-06-01 13:00:00          724696 2022-06-02 02:00:00  2177.855469           ft^3/s    streamflow   short_range       01013500

System Requirements

We employ several methods to make sure the resulting pandas.DataFrame produced by nwm_client are as efficient and manageable as possible. Nonetheless, this package can potentially use a large amount of memory.

The National Water Model generates multiple forecasts per day at over 3.7 million locations across the United States. A single forecast could be spread across hundreds of files and require repeated calls to the data source. The intermediate steps of retrieving and processing these files into leaner DataFrame may use several GB of memory. As such, recommended minimum requirements to use this package are a 4-core consumer processor and 8 GB of RAM.

Development

This package uses a setup configuration file (setup.cfg) and assumes use of the setuptools backend to build the package. To install the package for development use:

$ python3 -m venv env
$ source env/bin/activate
$ python3 -m pip install -U pip
$ python3 -m pip install -U setuptools
$ python3 -m pip install -e .[develop]

To generate a source distribution:

$ python3 -m pip install -U wheel build
$ python3 -m build

The packages generated in dist/ can be installed directly with pip or uploaded to PyPI using twine.

hydrotools.nwm-client dependencies

adlfs aiofiles aiohttp azure-storage-blob beautifulsoup4 dask google-cloud-storage hydrotools-restclient netcdf4 pandas pint planetary-computer pyarrow tables xarray

FAQs

What is hydrotools.nwm-client?

Retrieve National Water Model data from various sources. Visit Snyk Advisor to see a full health score report for hydrotools.nwm-client, including popularity, security, maintenance & community analysis.

Is hydrotools.nwm-client popular?

The python package hydrotools.nwm-client receives a total of 64 weekly downloads. As such, hydrotools.nwm-client popularity was classified as limited. Visit the popularity section on Snyk Advisor to see the full health analysis.

Is hydrotools.nwm-client well maintained?

We found that hydrotools.nwm-client demonstrated a healthy version release cadence and project activity. It has a community of 6 open source contributors collaborating on the project. See the full package health analysis to learn more about the package maintenance status.

Is hydrotools.nwm-client safe to use?

The python package hydrotools.nwm-client was scanned for known vulnerabilities and missing license, and no issues were found. Thus the package was deemed as safe to use. See the full health analysis review.

Last updated on 19 December-2024, at 09:52 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP

Package Health Score

Direct Usage Popularity

Security and license risk for latest version

Commit Frequency

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Snyk Vulnerability Scanner

Readme

OWPHydroTools :: NWM Client

Installation

Usage

Code

Example output

Example output

Example output

Example output

Example output

System Requirements

Development

hydrotools.nwm-client dependencies

FAQs

What is hydrotools.nwm-client?

Is hydrotools.nwm-client popular?

Is hydrotools.nwm-client well maintained?

Is hydrotools.nwm-client safe to use?

Build a secure application checklist

Select a recommended open source package

Scan your app for vulnerabilities

Source Code

Open Source

Container

Config

Fix identified vulnerabilities

Monitor for new issues