Package Health Score

67 / 100

security
No known security issues
popularity
Limited
maintenance
Healthy
community
Limited

Explore Similar Packages

Security

No known security issues

All security vulnerabilities belong to production dependencies of direct and indirect packages.

Security and license risk for significant versions

Version				Vulnerabilities	License Risk
2.2.0	\|	11/2024	Popular	0 C 0 H 0 M 0 L	0 H 0 M 0 L
2.1.0	\|	07/2024		0 C 0 H 0 M 0 L	0 H 0 M 0 L
1.2.0	\|	02/2024		0 C 0 H 0 M 0 L	0 H 0 M 0 L
1.0.0	\|	01/2024		0 C 0 H 0 M 0 L	0 H 0 M 0 L
1.1.0	\|	01/2024		0 C 0 H 0 M 0 L	0 H 0 M 0 L

License

MIT

Security Policy

Is your project affected by vulnerabilities?

Scan your projects for vulnerabilities. Fix quickly with automated fixes. Get started with Snyk for free.

Get started free

Popularity

Limited

Weekly Downloads (759)

GitHub Stars: 6
Forks: 3
Contributors: 4

Direct Usage Popularity

The npm package mistral-tokenizer-ts receives a total of 759 downloads a week. As such, we scored mistral-tokenizer-ts popularity level to be Limited.

Based on project statistics from the GitHub repository for the npm package mistral-tokenizer-ts, we found that it has been starred 6 times.

Downloads are calculated as moving averages for a period of the last 12 months, excluding weekends and known missing data points.

Community

Limited

Readme.md: Yes
Contributing.md: No
Code of Conduct: No
Contributors: 4
Funding: No

This project has seen only 10 or less contributors.

We found a way for you to contribute to the project! Looks like mistral-tokenizer-ts is missing a Code of Conduct.

Embed Package Health Score Badge

Maintenance

Healthy

Commit Frequency

Open Issues: 0
Open PR: 1
Last Release: 1 month ago
Last Commit: 1 month ago

Further analysis of the maintenance status of mistral-tokenizer-ts based on released npm versions cadence, the repository activity, and other data points determined that its maintenance is Healthy.

We found that mistral-tokenizer-ts demonstrates a positive version release cadence with at least one new version released in the past 3 months.

In the past month we didn't find any pull request activity or change in issues status has been detected for the GitHub repository.

Keep your project healthy

Check your package.json

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Node.js Compatibility: ^20

Age: 11 months
Dependencies: 1 Direct
Versions: 7
Install Size: 3.62 MB
Dist-tags: 1
# of Files: 29
Maintainers: 1
TS Typings: No

mistral-tokenizer-ts has more than a single and default latest tag published for the npm package. This means, there may be other tags available for this package, such as next to indicate future releases, or stable to indicate stable releases.

Readme

🌬️ mistral-tokenizer-ts 🌬️

Typescript tokenizer for Mistral models.

Supported models

Generalist models

mistral-large-latest (points to mistral-large-2407)
mistral-large-2402
mistral-large-2407
mistral-small-latest (points to mistral-small-2402)
mistral-small-2402
open-mistral-nemo (points to open-mistral-nemo-2407)
open-mistral-nemo-2407

Specialized models

codestral-latest (points to codestral-2405)
codestral-2405
mistral-embed (points to mistral-embed-2312)
mistral-embed-2312

Research models

open-mistral-7b (points to open-mistral-7b-v0.3)
open-mistral-7b-v0.1
open-mistral-7b-v0.2
open-mistral-7b-v0.3
open-mixtral-8x7b (points to open-mixtral-8x7b-v0.1)
open-mixtral-8x7b-v0.1
open-mixtral-8x22b (points to open-mixtral-8x22b-v0.1)
open-mixtral-8x22b-v0.1
open-codestral-mamba (points to open-codestral-mamba-v0.1)
open-codestral-mamba-v0.1

Install

npm install mistral-tokenizer-ts

Usage

import { getTokenizerForModel } from 'mistral-tokenizer-ts'

const tokenizer = getTokenizerForModel('open-mistral-7b')

// Encode.
const encoded = tokenizer.encode('Hello world!')

// Decode.
const decoded = tokenizer.decode([1, 22557, 1526])

Tests

npm run test

Credit

@imoneoi for the initial implementation
@dqbd for the tiktoken JS port
@mistralai for the Python tokenizers

mistral-tokenizer-ts dependencies

js-tiktoken

mistral-tokenizer-ts development dependencies

@types/jest @typescript-eslint/eslint-plugin @typescript-eslint/parser eslint eslint-config-prettier husky jest lint-staged microbundle prettier rimraf ts-jest typescript

FAQs

What is mistral-tokenizer-ts?

TS tokenizer for Mistral-based LLMs. Visit Snyk Advisor to see a full health score report for mistral-tokenizer-ts, including popularity, security, maintenance & community analysis.

Is mistral-tokenizer-ts popular?

The npm package mistral-tokenizer-ts receives a total of 759 weekly downloads. As such, mistral-tokenizer-ts popularity was classified as limited. Visit the popularity section on Snyk Advisor to see the full health analysis.

Is mistral-tokenizer-ts well maintained?

We found that mistral-tokenizer-ts demonstrated a healthy version release cadence and project activity. It has a community of 4 open source contributors collaborating on the project. See the full package health analysis to learn more about the package maintenance status.

Is mistral-tokenizer-ts safe to use?

The npm package mistral-tokenizer-ts was scanned for known vulnerabilities and missing license, and no issues were found. Thus the package was deemed as safe to use. See the full health analysis review.

Last updated on 12 December-2024, at 14:03 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP