diff --git a/README.md b/README.md index c4d1a22..738805a 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,46 @@ - node-fetch ========== -[![npm stable version][npm-image]][npm-url] -[![npm next version][npm-next-image]][npm-url] +[![npm version][npm-image]][npm-url] [![build status][travis-image]][travis-url] [![coverage status][codecov-image]][codecov-url] [![install size][install-size-image]][install-size-url] A light-weight module that brings `window.fetch` to Node.js + + +- [Motivation](#motivation) +- [Features](#features) +- [Difference from client-side fetch](#difference-from-client-side-fetch) +- [Installation](#installation) +- [Loading and configuring the module](#loading-and-configuring-the-module) +- [Common Usage](#common-usage) + - [Plain text or HTML](#plain-text-or-html) + - [JSON](#json) + - [Simple Post](#simple-post) + - [Post with JSON](#post-with-json) + - [Post with form parameters](#post-with-form-parameters) + - [Handling exceptions](#handling-exceptions) + - [Handling client and server errors](#handling-client-and-server-errors) +- [Advanced Usage](#advanced-usage) + - [Streams](#streams) + - [Buffer](#buffer) + - [Accessing Headers and other Meta data](#accessing-headers-and-other-meta-data) + - [Post data using a file stream](#post-data-using-a-file-stream) + - [Post with form-data (detect multipart)](#post-with-form-data-detect-multipart) +- [API](#api) + - [fetch(url[, options])](#fetchurl-options) + - [Options](#options) + - [Class: Request](#class-request) + - [Class: Response](#class-response) + - [Class: Headers](#class-headers) + - [Interface: Body](#interface-body) + - [Class: FetchError](#class-fetcherror) +- [License](#license) +- [Acknowledgement](#acknowledgement) + + ## Motivation @@ -17,171 +48,204 @@ Instead of implementing `XMLHttpRequest` in Node.js to run browser-specific [Fet See Matt Andrews' [isomorphic-fetch](https://github.com/matthew-andrews/isomorphic-fetch) or Leonardo Quixada's [cross-fetch](https://github.com/lquixada/cross-fetch) for isomorphic usage (exports `node-fetch` for server-side, `whatwg-fetch` for client-side). - ## Features - Stay consistent with `window.fetch` API. -- Make conscious trade-off when following [whatwg fetch spec][whatwg-fetch] and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known difference. +- Make conscious trade-off when following [WHATWG fetch spec][whatwg-fetch] and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known differences. - Use native promise, but allow substituting it with [insert your favorite promise library]. -- Use native stream for body, on both request and response. -- Decode content encoding (gzip/deflate) properly, convert `res.text()` output to UTF-8 optionally. -- Useful extensions such as timeout, redirect limit, response size limit, [explicit errors][ERROR-HANDLING.md] for troubleshooting. - +- Use native Node streams for body, on both request and response. +- Decode content encoding (gzip/deflate) properly, and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically. +- Useful extensions such as timeout, redirect limit, response size limit, [explicit errors](ERROR-HANDLING.md) for troubleshooting. ## Difference from client-side fetch -- See [Known Differences][LIMITS.md] for details. +- See [Known Differences](LIMITS.md) for details. - If you happen to use a missing feature that `window.fetch` offers, feel free to open an issue. - Pull requests are welcomed too! +## Installation -## Install - -Stable release (`2.x`) +Current stable release (`2.x`) ```sh $ npm install node-fetch --save ``` -## Usage +## Loading and configuring the module +We suggest you load the module via `require`, pending the stabalizing of es modules in node: +```js +const fetch = require('node-fetch'); +``` -Note that documentation below is up-to-date with `2.x` releases, [see `1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide][UPGRADE-GUIDE.md] if you want to find out the difference. +If you are using a Promise library other than native, set it through fetch.Promise: +```js +const Bluebird = require('bluebird'); -```javascript -import fetch from 'node-fetch'; -// or -// const fetch = require('node-fetch'); +fetch.Promise = Bluebird; +``` -// if you are using your own Promise library, set it through fetch.Promise. Eg. +## Common Usage -// import Bluebird from 'bluebird'; -// fetch.Promise = Bluebird; - -// plain text or html +NOTE: The documentation below is up-to-date with `2.x` releases, [see `1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide](UPGRADE-GUIDE.md) for the differences. +#### Plain text or HTML +```js fetch('https://github.com/') - .then(res => res.text()) - .then(body => console.log(body)); + .then(res => res.text()) + .then(body => console.log(body)); +``` -// json +#### JSON + +```js fetch('https://api.github.com/users/github') - .then(res => res.json()) - .then(json => console.log(json)); + .then(res => res.json()) + .then(json => console.log(json)); +``` -// catching network error -// 3xx-5xx responses are NOT network errors, and should be handled in then() -// you only need one catch() at the end of your promise chain +#### Simple Post +```js +fetch('https://httpbin.org/post', { method: 'POST', body: 'a=1' }) + .then(res => res.json()) // expecting a json response + .then(json => console.log(json)); +``` -fetch('http://domain.invalid/') - .catch(err => console.error(err)); +#### Post with JSON -// stream -// the node.js way is to use stream when possible +```js +const body = { a: 1 }; -fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') - .then(res => { - return new Promise((resolve, reject) => { - const dest = fs.createWriteStream('./octocat.png'); - res.body.pipe(dest); - res.body.on('error', err => { - reject(err); - }); - dest.on('finish', () => { - resolve(); - }); - dest.on('error', err => { - reject(err); - }); - }); - }); +fetch('https://httpbin.org/post', { + method: 'post', + body: JSON.stringify(body), + headers: { 'Content-Type': 'application/json' }, + }) + .then(res => res.json()) + .then(json => console.log(json)); +``` -// buffer -// if you prefer to cache binary data in full, use buffer() -// note that buffer() is a node-fetch only API +#### Post with form parameters +`URLSearchParams` is available in Node.js as of v7.5.0. See [official documentation](https://nodejs.org/api/url.html#url_class_urlsearchparams) for more usage methods. -import fileType from 'file-type'; +NOTE: The `Content-Type` header is only set automatically to `x-www-form-urlencoded` when an instance of `URLSearchParams` is given as such: -fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') - .then(res => res.buffer()) - .then(buffer => fileType(buffer)) - .then(type => { /* ... */ }); - -// meta - -fetch('https://github.com/') - .then(res => { - console.log(res.ok); - console.log(res.status); - console.log(res.statusText); - console.log(res.headers.raw()); - console.log(res.headers.get('content-type')); - }); - -// post - -fetch('http://httpbin.org/post', { method: 'POST', body: 'a=1' }) - .then(res => res.json()) - .then(json => console.log(json)); - -// post with stream from file - -import { createReadStream } from 'fs'; - -const stream = createReadStream('input.txt'); -fetch('http://httpbin.org/post', { method: 'POST', body: stream }) - .then(res => res.json()) - .then(json => console.log(json)); - -// post with JSON - -var body = { a: 1 }; -fetch('http://httpbin.org/post', { - method: 'POST', - body: JSON.stringify(body), - headers: { 'Content-Type': 'application/json' }, -}) - .then(res => res.json()) - .then(json => console.log(json)); - -// post form parameters (x-www-form-urlencoded) - -import { URLSearchParams } from 'url'; +```js +const { URLSearchParams } = require('url'); const params = new URLSearchParams(); params.append('a', 1); -fetch('http://httpbin.org/post', { method: 'POST', body: params }) - .then(res => res.json()) - .then(json => console.log(json)); -// post with form-data (detect multipart) +fetch('https://httpbin.org/post', { method: 'POST', body: params }) + .then(res => res.json()) + .then(json => console.log(json)); +``` -import FormData from 'form-data'; +#### Handling exceptions +NOTE: 3xx-5xx responses are *NOT* exceptions, and should be handled in `then()`, see the next section. + +Adding a catch to the fetch promise chain will catch *all* exceptions, such as errors originating from node core libraries, like network errors, and operational errors which are instances of FetchError. See the [error handling document](ERROR-HANDLING.md) for more details. + +```js +fetch('https://domain.invalid/') + .catch(err => console.error(err)); +``` + +#### Handling client and server errors +It is common to create a helper function to check that the response contains no client (4xx) or server (5xx) error responses: + +```js +function checkStatus(res) { + if (res.ok) { // res.status >= 200 && res.status < 300 + return res; + } else { + throw MyCustomError(res.statusText); + } +} + +fetch('https://httpbin.org/status/400') + .then(checkStatus) + .then(res => console.log('will not get here...')) +``` + +## Advanced Usage + +#### Streams +The "Node.js way" is to use streams when possible: + +```js +fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') + .then(res => { + const dest = fs.createWriteStream('./octocat.png'); + res.body.pipe(dest); + }); +``` + +[TODO]: # (Somewhere i think we also should mention arrayBuffer also if you want to be cross-fetch compatible.) + +#### Buffer +If you prefer to cache binary data in full, use buffer(). (NOTE: buffer() is a `node-fetch` only API) + +```js +const fileType = require('file-type'); + +fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png') + .then(res => res.buffer()) + .then(buffer => fileType(buffer)) + .then(type => { /* ... */ }); +``` + +#### Accessing Headers and other Meta data +```js +fetch('https://github.com/') + .then(res => { + console.log(res.ok); + console.log(res.status); + console.log(res.statusText); + console.log(res.headers.raw()); + console.log(res.headers.get('content-type')); + }); +``` + +#### Post data using a file stream + +```js +const { createReadStream } = require('fs'); + +const stream = createReadStream('input.txt'); + +fetch('https://httpbin.org/post', { method: 'POST', body: stream }) + .then(res => res.json()) + .then(json => console.log(json)); +``` + +#### Post with form-data (detect multipart) + +```js +const FormData = require('form-data'); const form = new FormData(); form.append('a', 1); -fetch('http://httpbin.org/post', { method: 'POST', body: form }) - .then(res => res.json()) - .then(json => console.log(json)); -// post with form-data (custom headers) -// note that getHeaders() is non-standard API +fetch('https://httpbin.org/post', { method: 'POST', body: form }) + .then(res => res.json()) + .then(json => console.log(json)); -import FormData from 'form-data'; +// OR, using custom headers +// NOTE: getHeaders() is non-standard API const form = new FormData(); form.append('a', 1); -fetch('http://httpbin.org/post', { method: 'POST', body: form, headers: form.getHeaders() }) - .then(res => res.json()) - .then(json => console.log(json)); -// node 7+ with async function +const options = { + method: 'POST', + body: form, + headers: form.getHeaders() +} -(async function () { - const res = await fetch('https://api.github.com/users/github'); - const json = await res.json(); - console.log(json); -})(); +fetch('https://httpbin.org/post', options) + .then(res => res.json()) + .then(json => console.log(json)); ``` See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js) for more examples. @@ -197,27 +261,29 @@ See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js) Perform an HTTP(S) fetch. -`url` should be an absolute url, such as `http://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected promise. +`url` should be an absolute url, such as `https://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected promise. + +[TODO]: # (It might be a good idea to reformat the options section into a table layout, like the headers section, instead of current code block.) -#### Options +### Options The default values are shown after each option key. ```js { - // These properties are part of the Fetch Standard - method: 'GET', - headers: {}, // request headers. format is the identical to that accepted by the Headers constructor (see below) - body: null, // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream - redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect + // These properties are part of the Fetch Standard + method: 'GET', + headers: {}, // request headers. format is the identical to that accepted by the Headers constructor (see below) + body: null, // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream + redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect - // The following properties are node-fetch extensions - follow: 20, // maximum redirect count. 0 to not follow redirect - timeout: 0, // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies) - compress: true, // support gzip/deflate content encoding. false to disable - size: 0, // maximum response body size in bytes. 0 to disable - agent: null // http(s).Agent instance, allows custom proxy, certificate, lookup, family etc. + // The following properties are node-fetch extensions + follow: 20, // maximum redirect count. 0 to not follow redirect + timeout: 0, // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies) + compress: true, // support gzip/deflate content encoding. false to disable + size: 0, // maximum response body size in bytes. 0 to disable + agent: null // http(s).Agent instance, allows custom proxy, certificate etc. } ``` @@ -225,6 +291,8 @@ The default values are shown after each option key. If no values are set, the following request headers will be sent automatically: +[TODO]: # ("we always said content-length will be "automatically calculated, if possible" in the default header section, but we never explain what's the condition for it to be calculated, and that chunked transfer-encoding will be used when they are not calculated or supplied." - "Maybe also add Transfer-Encoding: chunked? That header is added by Node.js automatically if the input is a stream, I believe.") + Header | Value ----------------- | -------------------------------------------------------- `Accept-Encoding` | `gzip,deflate` _(when `options.compress === true`)_ @@ -296,6 +364,8 @@ Because Node.js does not implement service workers (for which this class was des #### response.ok +*(spec-compliant)* + Convenience property representing if the request ended normally. Will evaluate to true if the response status was greater than or equal to 200 but smaller than 300. @@ -379,6 +449,8 @@ Consume the body and return a promise that will resolve to one of these formats. Consume the body and return a promise that will resolve to a Buffer. +[TODO]: # (textConverted API should mention an optional dependency on encoding, which users need to install by themselves, and this is done purely for backward compatibility with 1.x release.) + #### body.textConverted() *(node-fetch extension)* @@ -394,18 +466,15 @@ Identical to `body.text()`, except instead of always converting to UTF-8, encodi An operational error in the fetching process. See [ERROR-HANDLING.md][] for more info. -## License - -MIT - - ## Acknowledgement Thanks to [github/fetch](https://github.com/github/fetch) for providing a solid implementation reference. +## License + +MIT [npm-image]: https://img.shields.io/npm/v/node-fetch.svg?style=flat-square -[npm-next-image]: https://img.shields.io/npm/v/node-fetch/next.svg?style=flat-square [npm-url]: https://www.npmjs.com/package/node-fetch [travis-image]: https://img.shields.io/travis/bitinn/node-fetch.svg?style=flat-square [travis-url]: https://travis-ci.org/bitinn/node-fetch @@ -413,11 +482,10 @@ Thanks to [github/fetch](https://github.com/github/fetch) for providing a solid [codecov-url]: https://codecov.io/gh/bitinn/node-fetch [install-size-image]: https://packagephobia.now.sh/badge?p=node-fetch [install-size-url]: https://packagephobia.now.sh/result?p=node-fetch - -[ERROR-HANDLING.md]: https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md -[LIMITS.md]: https://github.com/bitinn/node-fetch/blob/master/LIMITS.md -[UPGRADE-GUIDE.md]: https://github.com/bitinn/node-fetch/blob/master/UPGRADE-GUIDE.md - [whatwg-fetch]: https://fetch.spec.whatwg.org/ [response-init]: https://fetch.spec.whatwg.org/#responseinit [node-readable]: https://nodejs.org/api/stream.html#stream_readable_streams +[mdn-headers]: https://developer.mozilla.org/en-US/docs/Web/API/Headers +[LIMITS.md]: https://github.com/bitinn/node-fetch/blob/master/LIMITS.md +[ERROR-HANDLING.md]: https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md +[UPGRADE-GUIDE.md]: https://github.com/bitinn/node-fetch/blob/master/UPGRADE-GUIDE.md \ No newline at end of file