How to use @postlight/mercury-parser - 10 common examples

To help you get started, we’ve selected a few @postlight/mercury-parser examples, based on popular ways it is used in public projects.

Secure your code as it's written. Use Snyk Code to scan source code in minutes - no build needed - and fix issues immediately.

github rnegron / noticias-pr-cli / cli.js View on Github external
async function retrieveArticleData(article) {
  const prepareArticleSpinner = ora('Cargando artículo...');

  // Get article data from Mercury
  try {
    prepareArticleSpinner.start();
    const articleData = await Mercury.parse(article);
    logger('Article Data: %O', articleData);

    prepareArticleSpinner.succeed();

    return articleData;
  } catch (err) {
    prepareArticleSpinner.fail(err.message);
    process.exit(1);
  }
}
github DIYgod / RSSHub / lib / middleware / parameter.js View on Github external
const parsed_result = await ctx.cache.tryGet(`mercury-cache-${link}`, async () => {
                        // if parser failed, return default description and not report error
                        try {
                            const res = await got(link);
                            const $ = cheerio.load(res.data);
                            const result = await mercury_parser.parse(link, {
                                html: $.html(),
                            });
                            return result;
                        } catch (e) {
                            // no-empty
                        }
                    });
github alltogethernow / web / server / index.js View on Github external
app.get('/api/parse/:url', (req, res) => {
  const Mercury = require('@postlight/mercury-parser')
  const url = decodeURIComponent(req.params.url)
  Mercury.parse(url)
    .then(result => {
      return res.json(result)
    })
    .catch(err => {
      console.log(`Error parsing article at ${url}:`, err)
      return res.status(500).json({ error: 'Error parsing article' })
    })
})
github IUNetSci / hoaxy-backend / hoaxy / node_scripts / parse_with_mercury.js View on Github external
#!/usr/bin/env node
process.env.UV_THREADPOOL_SIZE = 128;

const Mercury = require('@postlight/mercury-parser');

url = process.argv[2];
Mercury.parse(url).then((result) => {console.log(result)},(error) => {console.log("{'error': 'error'}")});
github feedbin / extract / app / app.js View on Github external
new validator(user, url, signature).validate().then(result => {
            mercury.parse(url).then(result => {
                const code = ("error" in result ? 400 : 200);
                response.status(code).send(result);
            }).catch(function(error) {
              response.status(400).json({ error: true, messages: "Cannot extract this URL." });
            });
        }).catch(function(error) {
            response.status(400).json({ error: true, messages: error });
github GetStream / Winds / api / src / parsers / content.js View on Github external
export async function ParseContent(url) {
	return await Mercury.parse(url);
}
github postlight / mercury-parser-api / src / parse-html.js View on Github external
const parseHtml = async ({ body }, context, cb) => {
  const { url, html } = JSON.parse(body);

  const result = await Mercury.parse(url, { html });

  return cb(
    null,
    result
      ? corsSuccessResponse(result)
      : corsErrorResponse({ message: 'There was an error parsing that URL.' })
  );
};
github yarnaimo / vanilla-clipper / src / utils / mercury.ts View on Github external
async function parse(url: string, html: string) {
    const result = Mercury.parse(url, {
        html,
        contentType: 'text',
        fetchAllPages: false,
    }) as MercuryResult

    if (!result.content) {
        return left(new Error('Parse error'))
    }

    return right(result)
}
github hello-efficiency-inc / raven-reader / src / renderer / parsers / article.js View on Github external
export async function parseArticle (url) {
  const result = await Mercury.parse(url)
  return result
}
github albertincx / formatbot1 / src / api / utils / ivMaker.js View on Github external
const parse = async (userUrl, paramsObj) => {
  const parseHelper = new ParseHelper(userUrl, paramsObj);
  userUrl = parseHelper.link;
  const opts = {};
  if (parseHelper.custom) {
    const html = await parseHelper.fetchHtml();
    logger(html, 'fixedFetched.html');
    opts.html = Buffer.from(html);
  }
  const extractor = parseHelper.getExtractor();
  if (extractor) {
    logger(extractor);
    Mercury.addExtractor(extractor);
  }
  let result = await mercury(userUrl, opts);
  logger(result.content, 'mercury.html');
  let { content } = result;
  const preContent = sanitizeHtml(content).trim();
  logger(preContent, 'preContent.html');
  if (preContent.length === 0) {
    const html = await parseHelper.puppet(userUrl);
    if (html) {
      logger(html, 'asyncContent.html');
      result = await mercury(userUrl, { html: Buffer.from(html) });
      logger(result.content, 'mercuryAsyncContent.html');
    }
  }
  let { title = '', url: source, iframe } = result;
  logger(iframe, 'iframes.html');

@postlight/mercury-parser

Mercury transforms web pages into clean text. Publishers and programmers use it to make the web make sense, and readers use it to read any web article comfortably.

Apache-2.0
Latest version published 3 years ago

Package Health Score

56 / 100
Full package analysis