From 745a27c38953129ce81f162b23c3d24f4a4d18f3 Mon Sep 17 00:00:00 2001 From: Jared Kantrowitz Date: Sat, 1 Sep 2018 07:23:02 -0400 Subject: [PATCH] README update (#504) * v2.x readme overhaul with additions discussed in #448 added "comments" (TODO link references) for changes suggested but not yet implemented for future discussion/prs clarified "native stream" to be "native Node streams" adjusted all uses of http to https to encourage secure protocol usage adjusted whatwg to proper case, WHATWG made code block tags consistent as `js` instead of `javascript` uppercased all method option values (post vs POST) added spec-compliant node to the `response.ok` api section * fix left over cruft, inconsistent hierarchy --- README.md | 358 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 213 insertions(+), 145 deletions(-) diff --git a/README.md b/README.md index c4d1a22..738805a 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,46 @@ - node-fetch ========== -[![npm stable version][npm-image]][npm-url] -[![npm next version][npm-next-image]][npm-url] +[![npm version][npm-image]][npm-url] [![build status][travis-image]][travis-url] [![coverage status][codecov-image]][codecov-url] [![install size][install-size-image]][install-size-url] A light-weight module that brings `window.fetch` to Node.js + + +- [Motivation](#motivation) +- [Features](#features) +- [Difference from client-side fetch](#difference-from-client-side-fetch) +- [Installation](#installation) +- [Loading and configuring the module](#loading-and-configuring-the-module) +- [Common Usage](#common-usage) + - [Plain text or HTML](#plain-text-or-html) + - [JSON](#json) + - [Simple Post](#simple-post) + - [Post with JSON](#post-with-json) + - [Post with form parameters](#post-with-form-parameters) + - [Handling exceptions](#handling-exceptions) + - [Handling client and server errors](#handling-client-and-server-errors) +- [Advanced Usage](#advanced-usage) + - [Streams](#streams) + - [Buffer](#buffer) + - [Accessing Headers and other Meta data](#accessing-headers-and-other-meta-data) + - [Post data using a file stream](#post-data-using-a-file-stream) + - [Post with form-data (detect multipart)](#post-with-form-data-detect-multipart) +- [API](#api) + - [fetch(url[, options])](#fetchurl-options) + - [Options](#options) + - [Class: Request](#class-request) + - [Class: Response](#class-response) + - [Class: Headers](#class-headers) + - [Interface: Body](#interface-body) + - [Class: FetchError](#class-fetcherror) +- [License](#license) +- [Acknowledgement](#acknowledgement) + + ## Motivation @@ -17,171 +48,204 @@ Instead of implementing `XMLHttpRequest` in Node.js to run browser-specific [Fet See Matt Andrews' [isomorphic-fetch](https://github.com/matthew-andrews/isomorphic-fetch) or Leonardo Quixada's [cross-fetch](https://github.com/lquixada/cross-fetch) for isomorphic usage (exports `node-fetch` for server-side, `whatwg-fetch` for client-side). - ## Features - Stay consistent with `window.fetch` API. -- Make conscious trade-off when following [whatwg fetch spec][whatwg-fetch] and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known difference. +- Make conscious trade-off when following [WHATWG fetch spec][whatwg-fetch] and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known differences. - Use native promise, but allow substituting it with [insert your favorite promise library]. -- Use native stream for body, on both request and response. -- Decode content encoding (gzip/deflate) properly, convert `res.text()` output to UTF-8 optionally. -- Useful extensions such as timeout, redirect limit, response size limit, [explicit errors][ERROR-HANDLING.md] for troubleshooting. - +- Use native Node streams for body, on both request and response. +- Decode content encoding (gzip/deflate) properly, and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically. +- Useful extensions such as timeout, redirect limit, response size limit, [explicit errors](ERROR-HANDLING.md) for troubleshooting. ## Difference from client-side fetch -- See [Known Differences][LIMITS.md] for details. +- See [Known Differences](LIMITS.md) for details. - If you happen to use a missing feature that `window.fetch` offers, feel free to open an issue. - Pull requests are welcomed too! +## Installation -## Install - -Stable release (`2.x`) +Current stable release (`2.x`) ```sh $ npm install node-fetch --save ``` -## Usage +## Loading and configuring the module +We suggest you load the module via `require`, pending the stabalizing of es modules in node: +```js +const fetch = require('node-fetch'); +``` -Note that documentation below is up-to-date with `2.x` releases, [see `1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide][UPGRADE-GUIDE.md] if you want to find out the difference. +If you are using a Promise library other than native, set it through fetch.Promise: +```js +const Bluebird = require('bluebird'); -```javascript -import fetch from 'node-fetch'; -// or -// const fetch = require('node-fetch'); +fetch.Promise = Bluebird; +``` -// if you are using your own Promise library, set it through fetch.Promise. Eg. +## Common Usage -// import Bluebird from 'bluebird'; -// fetch.Promise = Bluebird; - -// plain text or html +NOTE: The documentation below is up-to-date with `2.x` releases, [see `1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide](UPGRADE-GUIDE.md) for the differences. +#### Plain text or HTML +```js fetch('https://github.com/') - .then(res => res.text()) - .then(body => console.log(body)); + .then(res => res.text()) + .then(body => console.log(body)); +``` -// json +#### JSON + +```js fetch('https://api.github.com/users/github') - .then(res => res.json()) - .then(json => console.log(json)); + .then(res => res.json()) + .then(json => console.log(json)); +``` -// catching network error -// 3xx-5xx responses are NOT network errors, and should be handled in then() -// you only need one catch() at the end of your promise chain +#### Simple Post +```js +fetch('https://httpbin.org/post', { method: 'POST', body: 'a=1' }) + .then(res => res.json()) // expecting a json response + .then(json => console.log(json)); +``` -fetch('http://domain.invalid/') - .catch(err => console.error(err)); +#### Post with JSON -// stream -// the node.js way is to use stream when possible +```js +const body = { a: 1 }; -fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') - .then(res => { - return new Promise((resolve, reject) => { - const dest = fs.createWriteStream('./octocat.png'); - res.body.pipe(dest); - res.body.on('error', err => { - reject(err); - }); - dest.on('finish', () => { - resolve(); - }); - dest.on('error', err => { - reject(err); - }); - }); - }); +fetch('https://httpbin.org/post', { + method: 'post', + body: JSON.stringify(body), + headers: { 'Content-Type': 'application/json' }, + }) + .then(res => res.json()) + .then(json => console.log(json)); +``` -// buffer -// if you prefer to cache binary data in full, use buffer() -// note that buffer() is a node-fetch only API +#### Post with form parameters +`URLSearchParams` is available in Node.js as of v7.5.0. See [official documentation](https://nodejs.org/api/url.html#url_class_urlsearchparams) for more usage methods. -import fileType from 'file-type'; +NOTE: The `Content-Type` header is only set automatically to `x-www-form-urlencoded` when an instance of `URLSearchParams` is given as such: -fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') - .then(res => res.buffer()) - .then(buffer => fileType(buffer)) - .then(type => { /* ... */ }); - -// meta - -fetch('https://github.com/') - .then(res => { - console.log(res.ok); - console.log(res.status); - console.log(res.statusText); - console.log(res.headers.raw()); - console.log(res.headers.get('content-type')); - }); - -// post - -fetch('http://httpbin.org/post', { method: 'POST', body: 'a=1' }) - .then(res => res.json()) - .then(json => console.log(json)); - -// post with stream from file - -import { createReadStream } from 'fs'; - -const stream = createReadStream('input.txt'); -fetch('http://httpbin.org/post', { method: 'POST', body: stream }) - .then(res => res.json()) - .then(json => console.log(json)); - -// post with JSON - -var body = { a: 1 }; -fetch('http://httpbin.org/post', { - method: 'POST', - body: JSON.stringify(body), - headers: { 'Content-Type': 'application/json' }, -}) - .then(res => res.json()) - .then(json => console.log(json)); - -// post form parameters (x-www-form-urlencoded) - -import { URLSearchParams } from 'url'; +```js +const { URLSearchParams } = require('url'); const params = new URLSearchParams(); params.append('a', 1); -fetch('http://httpbin.org/post', { method: 'POST', body: params }) - .then(res => res.json()) - .then(json => console.log(json)); -// post with form-data (detect multipart) +fetch('https://httpbin.org/post', { method: 'POST', body: params }) + .then(res => res.json()) + .then(json => console.log(json)); +``` -import FormData from 'form-data'; +#### Handling exceptions +NOTE: 3xx-5xx responses are *NOT* exceptions, and should be handled in `then()`, see the next section. + +Adding a catch to the fetch promise chain will catch *all* exceptions, such as errors originating from node core libraries, like network errors, and operational errors which are instances of FetchError. See the [error handling document](ERROR-HANDLING.md) for more details. + +```js +fetch('https://domain.invalid/') + .catch(err => console.error(err)); +``` + +#### Handling client and server errors +It is common to create a helper function to check that the response contains no client (4xx) or server (5xx) error responses: + +```js +function checkStatus(res) { + if (res.ok) { // res.status >= 200 && res.status < 300 + return res; + } else { + throw MyCustomError(res.statusText); + } +} + +fetch('https://httpbin.org/status/400') + .then(checkStatus) + .then(res => console.log('will not get here...')) +``` + +## Advanced Usage + +#### Streams +The "Node.js way" is to use streams when possible: + +```js +fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') + .then(res => { + const dest = fs.createWriteStream('./octocat.png'); + res.body.pipe(dest); + }); +``` + +[TODO]: # (Somewhere i think we also should mention arrayBuffer also if you want to be cross-fetch compatible.) + +#### Buffer +If you prefer to cache binary data in full, use buffer(). (NOTE: buffer() is a `node-fetch` only API) + +```js +const fileType = require('file-type'); + +fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') + .then(res => res.buffer()) + .then(buffer => fileType(buffer)) + .then(type => { /* ... */ }); +``` + +#### Accessing Headers and other Meta data +```js +fetch('https://github.com/') + .then(res => { + console.log(res.ok); + console.log(res.status); + console.log(res.statusText); + console.log(res.headers.raw()); + console.log(res.headers.get('content-type')); + }); +``` + +#### Post data using a file stream + +```js +const { createReadStream } = require('fs'); + +const stream = createReadStream('input.txt'); + +fetch('https://httpbin.org/post', { method: 'POST', body: stream }) + .then(res => res.json()) + .then(json => console.log(json)); +``` + +#### Post with form-data (detect multipart) + +```js +const FormData = require('form-data'); const form = new FormData(); form.append('a', 1); -fetch('http://httpbin.org/post', { method: 'POST', body: form }) - .then(res => res.json()) - .then(json => console.log(json)); -// post with form-data (custom headers) -// note that getHeaders() is non-standard API +fetch('https://httpbin.org/post', { method: 'POST', body: form }) + .then(res => res.json()) + .then(json => console.log(json)); -import FormData from 'form-data'; +// OR, using custom headers +// NOTE: getHeaders() is non-standard API const form = new FormData(); form.append('a', 1); -fetch('http://httpbin.org/post', { method: 'POST', body: form, headers: form.getHeaders() }) - .then(res => res.json()) - .then(json => console.log(json)); -// node 7+ with async function +const options = { + method: 'POST', + body: form, + headers: form.getHeaders() +} -(async function () { - const res = await fetch('https://api.github.com/users/github'); - const json = await res.json(); - console.log(json); -})(); +fetch('https://httpbin.org/post', options) + .then(res => res.json()) + .then(json => console.log(json)); ``` See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js) for more examples. @@ -197,27 +261,29 @@ See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js) Perform an HTTP(S) fetch. -`url` should be an absolute url, such as `http://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected promise. +`url` should be an absolute url, such as `https://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected promise. + +[TODO]: # (It might be a good idea to reformat the options section into a table layout, like the headers section, instead of current code block.) -#### Options +### Options The default values are shown after each option key. ```js { - // These properties are part of the Fetch Standard - method: 'GET', - headers: {}, // request headers. format is the identical to that accepted by the Headers constructor (see below) - body: null, // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream - redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect + // These properties are part of the Fetch Standard + method: 'GET', + headers: {}, // request headers. format is the identical to that accepted by the Headers constructor (see below) + body: null, // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream + redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect - // The following properties are node-fetch extensions - follow: 20, // maximum redirect count. 0 to not follow redirect - timeout: 0, // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies) - compress: true, // support gzip/deflate content encoding. false to disable - size: 0, // maximum response body size in bytes. 0 to disable - agent: null // http(s).Agent instance, allows custom proxy, certificate, lookup, family etc. + // The following properties are node-fetch extensions + follow: 20, // maximum redirect count. 0 to not follow redirect + timeout: 0, // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies) + compress: true, // support gzip/deflate content encoding. false to disable + size: 0, // maximum response body size in bytes. 0 to disable + agent: null // http(s).Agent instance, allows custom proxy, certificate etc. } ``` @@ -225,6 +291,8 @@ The default values are shown after each option key. If no values are set, the following request headers will be sent automatically: +[TODO]: # ("we always said content-length will be "automatically calculated, if possible" in the default header section, but we never explain what's the condition for it to be calculated, and that chunked transfer-encoding will be used when they are not calculated or supplied." - "Maybe also add Transfer-Encoding: chunked? That header is added by Node.js automatically if the input is a stream, I believe.") + Header | Value ----------------- | -------------------------------------------------------- `Accept-Encoding` | `gzip,deflate` _(when `options.compress === true`)_ @@ -296,6 +364,8 @@ Because Node.js does not implement service workers (for which this class was des #### response.ok +*(spec-compliant)* + Convenience property representing if the request ended normally. Will evaluate to true if the response status was greater than or equal to 200 but smaller than 300. @@ -379,6 +449,8 @@ Consume the body and return a promise that will resolve to one of these formats. Consume the body and return a promise that will resolve to a Buffer. +[TODO]: # (textConverted API should mention an optional dependency on encoding, which users need to install by themselves, and this is done purely for backward compatibility with 1.x release.) + #### body.textConverted() *(node-fetch extension)* @@ -394,18 +466,15 @@ Identical to `body.text()`, except instead of always converting to UTF-8, encodi An operational error in the fetching process. See [ERROR-HANDLING.md][] for more info. -## License - -MIT - - ## Acknowledgement Thanks to [github/fetch](https://github.com/github/fetch) for providing a solid implementation reference. +## License + +MIT [npm-image]: https://img.shields.io/npm/v/node-fetch.svg?style=flat-square -[npm-next-image]: https://img.shields.io/npm/v/node-fetch/next.svg?style=flat-square [npm-url]: https://www.npmjs.com/package/node-fetch [travis-image]: https://img.shields.io/travis/bitinn/node-fetch.svg?style=flat-square [travis-url]: https://travis-ci.org/bitinn/node-fetch @@ -413,11 +482,10 @@ Thanks to [github/fetch](https://github.com/github/fetch) for providing a solid [codecov-url]: https://codecov.io/gh/bitinn/node-fetch [install-size-image]: https://packagephobia.now.sh/badge?p=node-fetch [install-size-url]: https://packagephobia.now.sh/result?p=node-fetch - -[ERROR-HANDLING.md]: https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md -[LIMITS.md]: https://github.com/bitinn/node-fetch/blob/master/LIMITS.md -[UPGRADE-GUIDE.md]: https://github.com/bitinn/node-fetch/blob/master/UPGRADE-GUIDE.md - [whatwg-fetch]: https://fetch.spec.whatwg.org/ [response-init]: https://fetch.spec.whatwg.org/#responseinit [node-readable]: https://nodejs.org/api/stream.html#stream_readable_streams +[mdn-headers]: https://developer.mozilla.org/en-US/docs/Web/API/Headers +[LIMITS.md]: https://github.com/bitinn/node-fetch/blob/master/LIMITS.md +[ERROR-HANDLING.md]: https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md +[UPGRADE-GUIDE.md]: https://github.com/bitinn/node-fetch/blob/master/UPGRADE-GUIDE.md \ No newline at end of file