Package Health Score

60 / 100

security
No known security issues
popularity
Limited
maintenance
Sustainable
community
Limited

Popularity

Limited

GitHub Stars: 0
Forks: 0
Contributors: 1

Direct Usage Popularity

Based on project statistics from the GitHub repository for the PyPI package ssc-codegen, we found that it has been starred ? times.

Security

No known security issues

Security and license risk for latest version

Release Date: Sep 29, 2024
Direct Vulnerabilities: 0
C
0
H
0
M
0
L
Indirect Vulnerabilities: 0
C
0
H
0
M
0
L
License Risk: 0
H
0
M
0
L

All security vulnerabilities belong to production dependencies of direct and indirect packages.

License

MIT

Security Policy

We found a way for you to contribute to the project! Looks like ssc-codegen is missing a security policy.

You can connect your project's repository to Snyk to stay up to date on security alerts and receive automatic fix pull requests.

Keep your project free of vulnerabilities with Snyk

Maintenance

Sustainable

Commit Frequency

Open Issues: 0
Open PR: 0
Last Release: 1 month ago
Last Commit: 3 months ago

Further analysis of the maintenance status of ssc-codegen based on released PyPI versions cadence, the repository activity, and other data points determined that its maintenance is Sustainable.

We found that ssc-codegen demonstrates a positive version release cadence with at least one new version released in the past 3 months.

In the past month we didn't find any pull request activity or change in issues status has been detected for the GitHub repository.

Community

Limited

Readme: Yes
Contributing.md: No
Code of Conduct: No
Contributors: 1
Funding: No

This project has seen only 10 or less contributors.

We found a way for you to contribute to the project! Looks like ssc-codegen is missing a Code of Conduct.

Embed Package Health Score Badge

Keep your project healthy

Check your requirements.txt

Ensure all the packages you're using are healthy and well-maintained

Snyk Vulnerability Scanner

Get health score & security insights directly in your IDE

Package

Python Versions Compatibility: >=3.10

Age: 12 months
Latest Release: 1 month ago
Dependencies: 0 Direct
Versions: 28
Maintainers: 1
Wheels: OS Independent

Readme

Selector Schema Codegen

RU EN

ssc_codegen is a generator for HTML parsers in various programming languages.

Why?

For convenient development of web scrapers, unofficial API interfaces, CI/CD integration
Support for API interfaces in various programming languages (currently available: Dart, Python)
Easy configuration files reading
auto documentation how to use it and generate parse structure signature
Portability: generated parsers are not tied to a specific project and can be reused
Simple syntax similar to jQuery, ORM frameworks, and data serialization libraries

Features

Declarative style: describe WHAT you want to do, not HOW to program it
Standardization: the generated code has minimal dependencies
Ability to rebuild in other programming languages
CSS, XPath, regex, minimal string formatting operations
Field validation, CSS/XPath/regex expressions
Documentation transfer into the generated code
Conversion of CSS to XPath queries

Install

pipx

pipx install ssc_codegen

pip

pip install ssc_codegen

Usage

See examples

Supported Libraries and Programming Languages

Language	Library	XPath Support	CSS Support	Formatter
Python	bs4	NO	YES	black
-	parsel	YES	YES	-
-	selectolax (modest)	NO	YES	-
-	scrapy (based on parsel, but class init argument - Response)	YES	YES	-
Dart	universal_html	NO	YES	dart format

Recommendations

For quickly obtaining effective CSS selectors, it is recommended to use any Chromium-based browser and the SelectorGadget extension.
Use CSS selectors: they can be guaranteed convert to XPath.
For maximum support across most programming languages, use simple queries for the following reasons:
- Some libraries do not support the full CSS specification. For example, the selector #product_description+ p works in python.parsel and javascript, but not in the dart.universal_html and selectolax libraries.
There is an XPath to CSS converter, but its functionality is not guaranteed. For example, CSS has no equivalent to contains from XPath.

How to Read Schema Code

Before reading, make sure you are familiar with:

CSS selectors
XPath selectors
Regular expressions

Shortcuts

Variable notations in the code:

D() — mark a Document/Element object
N() — mark operations with nested structures
R() — shortcut for D().raw(). Useful if you only need operations with regular expressions and strings, not with selectors

Built-in Schemas

ItemSchema

Parses the structure according to the rules { = , = , ...}, returns a hash table.

DictSchema

Parses the structure according to the rule { = , = , ...}, returns a hash table.

ListSchema

Parses the structure according to the rule [{ = , = , ...}, { = , = , ...}], returns a list of hash tables.

FlattenListSchema

Parses the structure according to the rule [, , ...], returns a list of objects.

Types

Currently, there are 5 types

TYPE	DESCRIPTION
DOCUMENT	1 element/object of the document. Always the first argument in the field
LIST_DOCUMENT	Collection of elements
STRING	Tag string/attribute/tag text
LIST_STRING	Collection of strings/attributes/text
NESTED	Collection of strings/attributes/text

Magic Methods

__SPLIT_DOC__ - splits the document into elements for easier parsing
__PRE_VALIDATE__ - pre-validation of the document using assert. Throws an error if validation fails
__KEY__, __VALUE__ - magic methods for initializing DictSchema structure
__ITEM__ - magic method for initializing FlattenListSchema structure

Operators

Method	Accepts	Returns	Example	Description
default(None/str)	None/str	DOCUMENT	`D().default(None)`	Default value if an error occurs. Must be the first
sub_parser	Schema	-	`N().sub_parser(Books)`	Passes the document/element to another parser object. Returns the obtained result
css	CSS query	DOCUMENT	`D().css('a')`	Returns the first found element of the selector result
xpath	XPATH query	DOCUMENT	`D().xpath('//a')`	Returns the first found element of the selector result
css_all	CSS query	LIST_DOCUMENT	`D().css_all('a')`	Returns all elements of the selector result
xpath_all	XPATH query	LIST_DOCUMENT	`D().xpath_all('//a')`	Returns all elements of the selector result
raw		STRING/LIST_STRING	`D().raw()`	Returns the raw HTML of the document/element. Works with DOCUMENT, LIST_DOCUMENT
text		STRING/LIST_STRING	`D().css('title').text()`	Returns the text from the HTML document/element. Works with DOCUMENT, LIST_DOCUMENT
attr	ATTR-NAME	STRING/LIST_STRING	`D().css('a').attr('href')`	Returns the attribute from the HTML tag. Works with DOCUMENT, LIST_DOCUMENT
trim	str	STRING/LIST_STRING	`R().trim('')`	Trims the string from the LEFT and RIGHT. Works with STRING, LIST_STRING
ltrim	str	STRING/LIST_STRING	`D().css('a').attr('href').ltrim('//')`	Trims the string from the LEFT. Works with STRING, LIST_STRING
rtrim	str	STRING/LIST_STRING	`D().css('title').rtrim(' ')`	Trims the string from the RIGHT. Works with STRING, LIST_STRING
replace/repl	old, new	STRING/LIST_STRING	`D().css('a').attr('href').repl('//', 'https://')`	Replaces the string. Works with STRING, LIST_STRING
format/fmt	template	STRING/LIST_STRING	`D().css('title').fmt("title: {{}}")`	Formats the string according to the template. Must have the `{{}}` marker. Works with STRING, LIST_STRING
re	pattern	STRING/LIST_STRING	`D().css('title').re('(\w+)')`	Finds the first matching result of the regex pattern. Works with STRING, LIST_STRING
re_all	pattern	LIST_STRING	`D().css('title').re('(\w+)')`	Finds all matching results of the regex pattern. Works with STRING
re_sub	pattern, repl	STRING/LIST_STRING	`D().css('title').re_sub('(\w+)', 'wow')`	Replaces the string according to the regex pattern. Works with STRING, LIST_STRING
index	int	STRING/DOCUMENT	`D().css_all('a').index(0)`	Takes the element by index. Works with LIST_DOCUMENT, LIST_STRING
first		-	`D().css_all('a').first`	Alias for index(0)
last		-	`D().css_all('a').last`	Alias for index(-1). Or implementation of a negative index
join	sep	STRING	`D().css_all('a').text().join(', ')`	Collects the collection into a string. Works with LIST_STRING
assert_in	str	NONE	`D().css_all('a').attr('href').assert_in('example.com')`	Checks if the string is in the collection. The checked argument must be LIST_STRING
assert_re	pattern	NONE	`D().css('a').attr('href').assert_re('example.com')`	Checks if the regex pattern is found. The checked argument must be STRING
assert_css	CSS query	NONE	`D().assert_css('title')`	Checks the element by CSS. The checked argument must be DOCUMENT
assert_xpath	XPATH query	NONE	`D().assert_xpath('//title')`	Checks the element by XPath. The checked argument must be DOCUMENT

FAQs

What is ssc-codegen?

generate web scrapers structures by dsl-like language based on python. Visit Snyk Advisor to see a full health score report for ssc-codegen, including popularity, security, maintenance & community analysis.

Is ssc-codegen popular?

The python package ssc-codegen receives a total of 538 weekly downloads. As such, ssc-codegen popularity was classified as limited. Visit the popularity section on Snyk Advisor to see the full health analysis.

Is ssc-codegen well maintained?

We found indications that ssc-codegen maintenance is sustainable demonstrating some project activity. We saw a total of 1 open source contributors collaborating on the project. See the full package health analysis to learn more about the package maintenance status.

Is ssc-codegen safe to use?

The python package ssc-codegen was scanned for known vulnerabilities and missing license, and no issues were found. Thus the package was deemed as safe to use. See the full health analysis review.

Last updated on 19 October-2024, at 10:05 (UTC).

Build a secure application checklist

Select a recommended open source package

Minimize your risk by selecting secure & well maintained open source packages

DONE

Scan your app for vulnerabilities

Scan your application to find vulnerabilities in your: source code, open source dependencies, containers and configuration files

SCAN NOW

Fix identified vulnerabilities

Easily fix your code by leveraging automatically generated PRs

AUTO FIX

Monitor for new issues

New vulnerabilities are discovered every day. Get notified if your application is affected

MONITOR APP