merge 3.x into master branch (#745)

* feat: Migrate TypeScript types (#669)

* style: Introduce linting via XO

* fix: Fix tests

* chore!: Drop support for nodejs 4 and 6

* chore: Fix Travis CI yml

* Use old Babel (needs migration)

* chore: lint everything

* chore: Migrate to microbundle

* Default response.statusText should be blank (#578)

* fix: Use correct AbortionError message

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Use modern @babel/register

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Remove redundant packages

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Readd form-data

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* fix: Fix tests and force utf8-encoded urls

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* lint index.js

* Update devDependencies & ignore `test` directory in linter options

* Remove unnecessary eslint-ignore comment

* Update the `lint` script to run linter on every file

* Remove unused const & unnecessary import

* TypeScript: Fix Body.blob() wrong type (DefinitelyTyped/DefinitelyTyped#33721)

* chore: Lint as part of the build process

* fix: Convert Content-Encoding to lowercase (#672)

* fix: Better object checks (#673)

* Fix stream piping (#670)

* chore: Remove useless check

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* style: Fix lint

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* style: Fix lint

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* refactor: Modernise code

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Ensure all files are properly included

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Update deps and utf8 should be in dependencies

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* test: Drop Node v4 from tests

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* test: Modernise code

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Move errors to seperate directory

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* refactor: Add fetch-blob (#678)

* feat: Migrate data uri integration

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* Allow setting custom highWaterMark via node-fetch options (#386) (#671)

* Expose highWaterMark option to body clone function

* Add highWaterMark to responseOptions

* Add highWaterMark as node-fetch-only option

* a way to silently pass highWaterMark to clone

* Chai helper

* Server helper

* Tests

* Remove debug comments

* Document highWaterMark option

* Add TypeScript types for the new highWaterMark option

* feat: Include system error in FetchError if one occurs (#654)

* style: Add editorconfig

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore!: Drop NodeJS v8

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Remove legacy code for node < 8

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Use proper checks for ArrayBuffer and AbortError

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Use explicitly set error name in checks

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* Propagate size and timeout to cloned response (#664)

* Remove --save option as it isn't required anymore (#581)

* Propagate size and timeout to cloned response


Co-authored-by: Steve Moser <contact@stevemoser.org>
Co-authored-by: Antoni Kepinski <xxczaki@pm.me>

* Update Response types

* Update devDependencies

* feat: Fallback to blob type (Closes: #607)

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* style: Update formatting

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* style: Fix linting issues

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* docs: Add info on patching the global object

* docs: Added non-globalThis polyfill

* Replace deprecated `url.resolve` with the new WHATWG URL

* Update devDependencies

* Format code in examples to use `xo` style

* Verify examples with RunKit and edit them if necessary

* Add information about TypeScript support

* Document the new `highWaterMark` option

* Add Discord badge & information about Open Collective

* Style change

* Edit acknowledgement & add "Team" section

* fix table

* Format example code to use xo style

* chore: v3 release changelog

* Add the recommended way to fix `highWaterMark` issues

* docs: Add simple Runkit example

* fix: Properly set the name of the errors.

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* docs: Add AbortError to documented types

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* docs: AbortError proper typing parameters

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* docs: Add example code for Runkit

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* Replace microbundle with @pika/pack (#689)

* gitignore the pkg/ directory

* Move TypeScript types to the root of the project

* Replace microbundle with @pika/pack

* chore: Remove @pika/plugin-build-web and revert ./dist output directory

Signed-off-by: Richie Bendall <richiebendall@gmail.com>


Co-authored-by: Richie Bendall <richiebendall@gmail.com>

* fix incorrect statement in changelog

* chore: v3.x upgrade guide

* Change the Open Collective button

* docs: Encode support button as Markdown instead of HTML

* chore: Ignore proper directory in xo

* Add an "Upgrading" section to readme

* Split the upgrade guide into 2 files & add the missing changes about v3.x

* style: Lint test and example files

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* Move *.md files to the `docs` folder (except README.md)

* Update references to files

* Split LIMITS.md into 2 files (as of v2.x and v3.x)

* chore: Remove logging statement

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* style: Fix lint

* docs: Correct typings for systemError in FetchError (Fixes #697)

* refactor: Replace `encoding` with `fetch-charset-detection`. (#694)

* refactor: Replace `encoding` with `fetch-charset-detection`.

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* refactor: Move writing to stream back to body.js

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* refactor: Only put convertBody in fetch-charset-detection and refactor others.

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* test: Readd tests for getTotalBytes and extractContentType

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Revert package.json indention

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Remove optional dependency

* docs: Replace code for fetch-charset-detection with documentation.

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Remove iconv-lite

* fix: Use default export instead of named export for convertBody

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Remove unneeded installation of fetch-charset-detection in the build

* docs: Fix typo

* fix: Throw SyntaxError instead of FetchError in case of invalid… (#700)

* fix: Throw SyntaxError instead of FetchError in case of invalid JSON

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* docs: Add to upgrade guide

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* Remove deprecated url.parse from test

* Remove deprecated url.parse from server

* fix: Proper data uri to buffer conversion (#703)

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Add funding info

* fix: Flawed property existence test (#706)

Fix a problem where not all prototype methods are copied from the Body via the mixin method due to a failure to properly detect properties in the target. The current code uses the `in` operator, which may return properties lower down the inheritance chain, thus causing them to fail the copy. The new code properly calls the `.hasOwnProperty()` method to make the determination.

* fix: Properly handle stream pipeline double-fire

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* docs: Fix spelling

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Add `funding` field to package.json (#708)

* Fix: Do not set ContentLength to NaN (#709)

* do not set ContentLength to NaN

* lint

* docs: Add logo

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Update repository name from bitinn/node-fetch to node-fetch/node-fetch.

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Fix unit tests

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore(deps): Bump @pika/plugin-copy-assets from 0.7.1 to 0.8.1 (#713)

Bumps [@pika/plugin-copy-assets](https://github.com/pikapkg/builders) from 0.7.1 to 0.8.1.
- [Release notes](https://github.com/pikapkg/builders/releases)
- [Commits](https://github.com/pikapkg/builders/compare/v0.7.1...v0.8.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* chore(deps): Bump @pika/plugin-build-types from 0.7.1 to 0.8.1 (#710)

Bumps [@pika/plugin-build-types](https://github.com/pikapkg/builders) from 0.7.1 to 0.8.1.
- [Release notes](https://github.com/pikapkg/builders/releases)
- [Commits](https://github.com/pikapkg/builders/compare/v0.7.1...v0.8.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump nyc from 14.1.1 to 15.0.0 (#714)

Bumps [nyc](https://github.com/istanbuljs/nyc) from 14.1.1 to 15.0.0.
- [Release notes](https://github.com/istanbuljs/nyc/releases)
- [Changelog](https://github.com/istanbuljs/nyc/blob/master/CHANGELOG.md)
- [Commits](https://github.com/istanbuljs/nyc/compare/v14.1.1...v15.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* chore: Update travis ci url

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore(deps): Bump mocha from 6.2.2 to 7.0.0 (#711)

Bumps [mocha](https://github.com/mochajs/mocha) from 6.2.2 to 7.0.0.
- [Release notes](https://github.com/mochajs/mocha/releases)
- [Changelog](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mochajs/mocha/compare/v6.2.2...v7.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* feat: Allow excluding a user agent in a fetch request by setting… (#715)

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* Bump @pika/plugin-build-node from 0.7.1 to 0.8.1 (#717)

Bumps [@pika/plugin-build-node](https://github.com/pikapkg/builders) from 0.7.1 to 0.8.1.
- [Release notes](https://github.com/pikapkg/builders/releases)
- [Commits](https://github.com/pikapkg/builders/compare/v0.7.1...v0.8.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump @pika/plugin-standard-pkg from 0.7.1 to 0.8.1 (#716)

Bumps [@pika/plugin-standard-pkg](https://github.com/pikapkg/builders) from 0.7.1 to 0.8.1.
- [Release notes](https://github.com/pikapkg/builders/releases)
- [Commits](https://github.com/pikapkg/builders/compare/v0.7.1...v0.8.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump form-data from 2.5.1 to 3.0.0 (#712)

Bumps [form-data](https://github.com/form-data/form-data) from 2.5.1 to 3.0.0.
- [Release notes](https://github.com/form-data/form-data/releases)
- [Commits](https://github.com/form-data/form-data/compare/v2.5.1...v3.0.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* fix: typo

* update suggestion

* feat: Added missing redirect function (#718)

* added missing redirect function
* chore: Add types

Co-authored-by: Richie Bendall <richiebendall@gmail.com>

* fix: Use req.setTimeout for timeout (#719)

* chore: Update typings comment

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* chore: Update deps

Signed-off-by: Richie Bendall <richiebendall@gmail.com>

* docs: center badges & Open Collective button

* docs: add missing comma

* Remove current stable & LTS node version numbers from the comments

I don't think we really want to update them

* Bump xo from 0.25.4 to 0.26.1 (#730)

Bumps [xo](https://github.com/xojs/xo) from 0.25.4 to 0.26.1.
- [Release notes](https://github.com/xojs/xo/releases)
- [Commits](https://github.com/xojs/xo/compare/v0.25.4...v0.26.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump @pika/plugin-build-types from 0.8.3 to 0.9.2 (#729)

Bumps [@pika/plugin-build-types](https://github.com/pikapkg/builders) from 0.8.3 to 0.9.2.
- [Release notes](https://github.com/pikapkg/builders/releases)
- [Commits](https://github.com/pikapkg/builders/compare/v0.8.3...v0.9.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Bump @pika/plugin-standard-pkg from 0.8.3 to 0.9.2 (#726)

Bumps [@pika/plugin-standard-pkg](https://github.com/pikapkg/builders) from 0.8.3 to 0.9.2.
- [Release notes](https://github.com/pikapkg/builders/releases)
- [Commits](https://github.com/pikapkg/builders/compare/v0.8.3...v0.9.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* docs: Update information about `req.body` type in v3.x release

* Add information about removed body type to the v3 upgrade guide

* add awesome badge

* Show 2 ways of importing node-fetch (CommonJS & ES module)

* update dependencies

* lint

* refactor: Replace `url.parse` with `new URL()` (#701)

* chore: replace `url.parse` with `new URL()`

* lint

* handle relative URLs

* Change error message

* detect whether the url is absolute or not

* update tests

* drop relative url support

* lint

* fix tests

* typo

* Add information about dropped arbitrary URL support in v3.x upgrade guide

* set xo linting rule (node/no-deprecated-api) to on

* remove the `utf8` dependency

* fix

* refactor: split tests into several files, create the `utils` directory

* Update package.json scripts & remove unnecessary xo linting rules

* refactor: turn on some xo linting rules to improve code quality

* fix tests

* Remove invalid urls

* fix merge conflict

* update the upgrade guide

* test if URLs are encoded as UTF-8

* update xo to 0.28.0

* chore: Build before publishing

* v3.0.0-beta.1

* fix lint on test/main.js

Co-authored-by: Richie Bendall <richiebendall@gmail.com>
Co-authored-by: Antoni Kepinski <xxczaki@pm.me>
Co-authored-by: aeb-sia <50743092+aeb-sia@users.noreply.github.com>
Co-authored-by: Nazar Mokrynskyi <nazar@mokrynskyi.com>
Co-authored-by: Steve Moser <contact@stevemoser.org>
Co-authored-by: Erick Calder <e@arix.com>
Co-authored-by: Yaacov Rydzinski <yaacovCR@gmail.com>
Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Jimmy Wärting <jimmy@warting.se>
This commit is contained in:
David Frank 2020-03-13 23:06:25 +08:00 committed by GitHub
parent cd33d22378
commit 0959ca9739
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
42 changed files with 2974 additions and 2484 deletions

View File

@ -1,52 +0,0 @@
{
env: {
test: {
presets: [
[ 'env', {
loose: true,
targets: { node: 4 },
exclude: [
// skip some almost-compliant features on Node.js v4.x
'transform-es2015-block-scoping',
'transform-es2015-classes',
'transform-es2015-for-of',
]
} ]
],
plugins: [
'./build/babel-plugin'
]
},
coverage: {
presets: [
[ 'env', {
loose: true,
targets: { node: 4 },
exclude: [
'transform-es2015-block-scoping',
'transform-es2015-classes',
'transform-es2015-for-of'
]
} ]
],
plugins: [
[ 'istanbul', { exclude: [ 'src/blob.js', 'build', 'test' ] } ],
'./build/babel-plugin'
]
},
rollup: {
presets: [
[ 'env', {
loose: true,
targets: { node: 4 },
exclude: [
'transform-es2015-block-scoping',
'transform-es2015-classes',
'transform-es2015-for-of'
],
modules: false
} ]
]
}
}
}

13
.editorconfig Normal file
View File

@ -0,0 +1,13 @@
# editorconfig.org
root = true
[*]
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
indent_style = tab
[*.md]
trim_trailing_whitespace = false

6
.gitignore vendored
View File

@ -1,3 +1,9 @@
# Sketch temporary file
~*.sketch
# Generated files
dist/
# Logs
logs
*.log

7
.nycrc
View File

@ -1,7 +0,0 @@
{
"require": [
"babel-register"
],
"sourceMap": false,
"instrument": false
}

View File

@ -1,20 +1,16 @@
language: node_js
node_js:
- "4"
- "6"
- "8"
- "10"
- "node"
env:
- FORMDATA_VERSION=1.0.0
- FORMDATA_VERSION=2.1.0
before_script:
- 'if [ "$FORMDATA_VERSION" ]; then npm install form-data@^$FORMDATA_VERSION; fi'
- "lts/*" # Latest LTS
- "node" # Latest Stable
matrix:
include:
- # Linting stage
node_js: "lts/*" # Latest LTS
script: npm run lint
cache: npm
script:
- npm uninstall encoding
- npm run coverage
- npm install encoding
- npm run coverage
cache:
directories:
- node_modules

533
README.md
View File

@ -1,15 +1,22 @@
node-fetch
==========
<div align="center">
<img src="docs/media/Banner.svg" alt="Node Fetch"/>
<br>
<p>A light-weight module that brings <code>window.fetch</code> to Node.js.</p>
<a href="https://travis-ci.com/node-fetch/node-fetch"><img src="https://img.shields.io/travis/com/node-fetch/node-fetch/master?style=flat-square" alt="Build status"></a>
<a href="https://codecov.io/gh/node-fetch/node-fetch"><img src="https://img.shields.io/codecov/c/gh/node-fetch/node-fetch/master?style=flat-square" alt="Coverage status"></a>
<a href="https://packagephobia.now.sh/result?p=node-fetch"><img src="https://flat.badgen.net/packagephobia/install/node-fetch" alt="Current version"></a>
<a href="https://www.npmjs.com/package/node-fetch"><img src="https://img.shields.io/npm/v/node-fetch?style=flat-square" alt="Install size"></a>
<a href="https://github.com/sindresorhus/awesome-nodejs"><img src="https://awesome.re/mentioned-badge-flat.svg" alt="Mentioned in Awesome Node.js"></a>
<a href="https://discord.gg/Zxbndcm"><img src="https://img.shields.io/discord/619915844268326952?color=%237289DA&label=Discord&style=flat-square" alt="Discord"></a>
<br>
<br>
<b>Consider supporting us on our Open Collective:</b>
<br>
<br>
<a href="https://opencollective.com/node-fetch"><img src="https://opencollective.com/node-fetch/donate/button.png?color=blue" alt="Open Collective"></a>
</div>
[![npm version][npm-image]][npm-url]
[![build status][travis-image]][travis-url]
[![coverage status][codecov-image]][codecov-url]
[![install size][install-size-image]][install-size-url]
[![Discord][discord-image]][discord-url]
A light-weight module that brings `window.fetch` to Node.js
(We are looking for [v2 maintainers and collaborators](https://github.com/bitinn/node-fetch/issues/567))
---
[![Backers][opencollective-image]][opencollective-url]
@ -20,6 +27,7 @@ A light-weight module that brings `window.fetch` to Node.js
- [Difference from client-side fetch](#difference-from-client-side-fetch)
- [Installation](#installation)
- [Loading and configuring the module](#loading-and-configuring-the-module)
- [Upgrading](#upgrading)
- [Common Usage](#common-usage)
- [Plain text or HTML](#plain-text-or-html)
- [JSON](#json)
@ -39,13 +47,32 @@ A light-weight module that brings `window.fetch` to Node.js
- [API](#api)
- [fetch(url[, options])](#fetchurl-options)
- [Options](#options)
- [Default Headers](#default-headers)
- [Custom Agent](#custom-agent)
- [Custom highWaterMark](#custom-highwatermark)
- [Class: Request](#class-request)
- [new Request(input[, options])](#new-requestinput-options)
- [Class: Response](#class-response)
- [new Response([body[, options]])](#new-responsebody-options)
- [response.ok](#responseok)
- [response.redirected](#responseredirected)
- [Class: Headers](#class-headers)
- [new Headers([init])](#new-headersinit)
- [Interface: Body](#interface-body)
- [body.body](#bodybody)
- [body.bodyUsed](#bodybodyused)
- [body.arrayBuffer()](#bodyarraybuffer)
- [body.blob()](#bodyblob)
- [body.json()](#bodyjson)
- [body.text()](#bodytext)
- [body.buffer()](#bodybuffer)
- [Class: FetchError](#class-fetcherror)
- [License](#license)
- [Class: AbortError](#class-aborterror)
- [TypeScript](#typescript)
- [Acknowledgement](#acknowledgement)
- [Team](#team)
- [Former](#former)
- [License](#license)
<!-- /TOC -->
@ -59,247 +86,314 @@ See Matt Andrews' [isomorphic-fetch](https://github.com/matthew-andrews/isomorph
- Stay consistent with `window.fetch` API.
- Make conscious trade-off when following [WHATWG fetch spec][whatwg-fetch] and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known differences.
- Use native promise but allow substituting it with [insert your favorite promise library].
- Use native Node streams for body on both request and response.
- Decode content encoding (gzip/deflate) properly and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically.
- Useful extensions such as timeout, redirect limit, response size limit, [explicit errors](ERROR-HANDLING.md) for troubleshooting.
- Use native promise, but allow substituting it with [insert your favorite promise library].
- Use native Node streams for body, on both request and response.
- Decode content encoding (gzip/deflate) properly, and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically.
- Useful extensions such as timeout, redirect limit, response size limit, [explicit errors][error-handling.md] for troubleshooting.
## Difference from client-side fetch
- See [Known Differences](LIMITS.md) for details.
- See known differences:
- [As of v3.x](docs/v3-LIMITS.md)
- [As of v2.x](docs/v2-LIMITS.md)
- If you happen to use a missing feature that `window.fetch` offers, feel free to open an issue.
- Pull requests are welcomed too!
## Installation
Current stable release (`2.x`)
Current stable release (`3.x`)
```sh
$ npm install node-fetch
```
## Loading and configuring the module
We suggest you load the module via `require` until the stabilization of ES modules in node:
```js
// CommonJS
const fetch = require('node-fetch');
// ES Module
import fetch from 'node-fetch';
```
If you are using a Promise library other than native, set it through `fetch.Promise`:
```js
const fetch = require('node-fetch');
const Bluebird = require('bluebird');
fetch.Promise = Bluebird;
```
If you want to patch the global object in node:
```js
const fetch = require('node-fetch');
if (!globalThis.fetch) {
globalThis.fetch = fetch;
}
```
For versions of node earlier than 12.x, use this `globalThis` [polyfill](https://mathiasbynens.be/notes/globalthis):
```js
(function() {
if (typeof globalThis === 'object') return;
Object.defineProperty(Object.prototype, '__magic__', {
get: function() {
return this;
},
configurable: true
});
__magic__.globalThis = __magic__;
delete Object.prototype.__magic__;
}());
```
## Upgrading
Using an old version of node-fetch? Check out the following files:
- [2.x to 3.x upgrade guide](docs/v3-UPGRADE-GUIDE.md)
- [1.x to 2.x upgrade guide](docs/v2-UPGRADE-GUIDE.md)
- [Changelog](docs/CHANGELOG.md)
## Common Usage
NOTE: The documentation below is up-to-date with `2.x` releases; see the [`1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide](UPGRADE-GUIDE.md) for the differences.
NOTE: The documentation below is up-to-date with `3.x` releases, if you are using an older version, please check how to [upgrade](#upgrading).
### Plain text or HTML
#### Plain text or HTML
```js
const fetch = require('node-fetch');
fetch('https://github.com/')
.then(res => res.text())
.then(body => console.log(body));
.then(res => res.text())
.then(body => console.log(body));
```
#### JSON
### JSON
```js
const fetch = require('node-fetch');
fetch('https://api.github.com/users/github')
.then(res => res.json())
.then(json => console.log(json));
.then(res => res.json())
.then(json => console.log(json));
```
#### Simple Post
### Simple Post
```js
fetch('https://httpbin.org/post', { method: 'POST', body: 'a=1' })
.then(res => res.json()) // expecting a json response
.then(json => console.log(json));
const fetch = require('node-fetch');
fetch('https://httpbin.org/post', {method: 'POST', body: 'a=1'})
.then(res => res.json()) // expecting a json response
.then(json => console.log(json));
```
#### Post with JSON
### Post with JSON
```js
const body = { a: 1 };
const fetch = require('node-fetch');
const body = {a: 1};
fetch('https://httpbin.org/post', {
method: 'post',
body: JSON.stringify(body),
headers: { 'Content-Type': 'application/json' },
})
.then(res => res.json())
.then(json => console.log(json));
method: 'post',
body: JSON.stringify(body),
headers: {'Content-Type': 'application/json'}
})
.then(res => res.json())
.then(json => console.log(json));
```
#### Post with form parameters
`URLSearchParams` is available in Node.js as of v7.5.0. See [official documentation](https://nodejs.org/api/url.html#url_class_urlsearchparams) for more usage methods.
### Post with form parameters
`URLSearchParams` is available on the global object in Node.js as of v10.0.0. See [official documentation](https://nodejs.org/api/url.html#url_class_urlsearchparams) for more usage methods.
NOTE: The `Content-Type` header is only set automatically to `x-www-form-urlencoded` when an instance of `URLSearchParams` is given as such:
```js
const { URLSearchParams } = require('url');
const fetch = require('node-fetch');
const params = new URLSearchParams();
params.append('a', 1);
fetch('https://httpbin.org/post', { method: 'POST', body: params })
.then(res => res.json())
.then(json => console.log(json));
fetch('https://httpbin.org/post', {method: 'POST', body: params})
.then(res => res.json())
.then(json => console.log(json));
```
#### Handling exceptions
NOTE: 3xx-5xx responses are *NOT* exceptions and should be handled in `then()`; see the next section for more information.
### Handling exceptions
Adding a catch to the fetch promise chain will catch *all* exceptions, such as errors originating from node core libraries, network errors and operational errors, which are instances of FetchError. See the [error handling document](ERROR-HANDLING.md) for more details.
NOTE: 3xx-5xx responses are _NOT_ exceptions, and should be handled in `then()`, see the next section.
Adding a catch to the fetch promise chain will catch _all_ exceptions, such as errors originating from node core libraries, like network errors, and operational errors which are instances of FetchError. See the [error handling document][error-handling.md] for more details.
```js
fetch('https://domain.invalid/')
.catch(err => console.error(err));
const fetch = require('node-fetch');
fetch('https://domain.invalid/').catch(err => console.error(err));
```
#### Handling client and server errors
### Handling client and server errors
It is common to create a helper function to check that the response contains no client (4xx) or server (5xx) error responses:
```js
const fetch = require('node-fetch');
function checkStatus(res) {
if (res.ok) { // res.status >= 200 && res.status < 300
return res;
} else {
throw MyCustomError(res.statusText);
}
if (res.ok) {
// res.status >= 200 && res.status < 300
return res;
} else {
throw MyCustomError(res.statusText);
}
}
fetch('https://httpbin.org/status/400')
.then(checkStatus)
.then(res => console.log('will not get here...'))
.then(checkStatus)
.then(res => console.log('will not get here...'));
```
## Advanced Usage
#### Streams
### Streams
The "Node.js way" is to use streams when possible:
```js
fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png')
.then(res => {
const dest = fs.createWriteStream('./octocat.png');
res.body.pipe(dest);
});
const {createWriteStream} = require('fs');
const fetch = require('node-fetch');
fetch(
'https://octodex.github.com/images/Fintechtocat.png'
).then(res => {
const dest = fs.createWriteStream('./octocat.png');
res.body.pipe(dest);
});
```
#### Buffer
If you prefer to cache binary data in full, use buffer(). (NOTE: `buffer()` is a `node-fetch`-only API)
### Buffer
If you prefer to cache binary data in full, use buffer(). (NOTE: buffer() is a `node-fetch` only API)
```js
const fetch = require('node-fetch');
const fileType = require('file-type');
fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png')
.then(res => res.buffer())
.then(buffer => fileType(buffer))
.then(type => { /* ... */ });
fetch('https://octodex.github.com/images/Fintechtocat.png')
.then(res => res.buffer())
.then(buffer => fileType(buffer))
.then(type => {
console.log(type);
});
```
#### Accessing Headers and other Meta data
### Accessing Headers and other Meta data
```js
fetch('https://github.com/')
.then(res => {
console.log(res.ok);
console.log(res.status);
console.log(res.statusText);
console.log(res.headers.raw());
console.log(res.headers.get('content-type'));
});
const fetch = require('node-fetch');
fetch('https://github.com/').then(res => {
console.log(res.ok);
console.log(res.status);
console.log(res.statusText);
console.log(res.headers.raw());
console.log(res.headers.get('content-type'));
});
```
#### Extract Set-Cookie Header
### Extract Set-Cookie Header
Unlike browsers, you can access raw `Set-Cookie` headers manually using `Headers.raw()`. This is a `node-fetch` only API.
```js
fetch(url).then(res => {
// returns an array of values, instead of a string of comma-separated values
console.log(res.headers.raw()['set-cookie']);
const fetch = require('node-fetch');
fetch('https://example.com').then(res => {
// returns an array of values, instead of a string of comma-separated values
console.log(res.headers.raw()['set-cookie']);
});
```
#### Post data using a file stream
### Post data using a file stream
```js
const { createReadStream } = require('fs');
const {createReadStream} = require('fs');
const fetch = require('node-fetch');
const stream = createReadStream('input.txt');
fetch('https://httpbin.org/post', { method: 'POST', body: stream })
.then(res => res.json())
.then(json => console.log(json));
fetch('https://httpbin.org/post', {method: 'POST', body: stream})
.then(res => res.json())
.then(json => console.log(json));
```
#### Post with form-data (detect multipart)
### Post with form-data (detect multipart)
```js
const fetch = require('node-fetch');
const FormData = require('form-data');
const form = new FormData();
form.append('a', 1);
fetch('https://httpbin.org/post', { method: 'POST', body: form })
.then(res => res.json())
.then(json => console.log(json));
fetch('https://httpbin.org/post', {method: 'POST', body: form})
.then(res => res.json())
.then(json => console.log(json));
// OR, using custom headers
// NOTE: getHeaders() is non-standard API
const form = new FormData();
form.append('a', 1);
const options = {
method: 'POST',
body: form,
headers: form.getHeaders()
}
method: 'POST',
body: form,
headers: form.getHeaders()
};
fetch('https://httpbin.org/post', options)
.then(res => res.json())
.then(json => console.log(json));
.then(res => res.json())
.then(json => console.log(json));
```
#### Request cancellation with AbortSignal
> NOTE: You may cancel streamed requests only on Node >= v8.0.0
### Request cancellation with AbortSignal
You may cancel requests with `AbortController`. A suggested implementation is [`abort-controller`](https://www.npmjs.com/package/abort-controller).
An example of timing out a request after 150ms could be achieved as the following:
```js
import AbortController from 'abort-controller';
const fetch = require('node-fetch');
const AbortController = require('abort-controller');
const controller = new AbortController();
const timeout = setTimeout(
() => { controller.abort(); },
150,
);
const timeout = setTimeout(() => {
controller.abort();
}, 150);
fetch(url, { signal: controller.signal })
.then(res => res.json())
.then(
data => {
useData(data)
},
err => {
if (err.name === 'AbortError') {
// request was aborted
}
},
)
.finally(() => {
clearTimeout(timeout);
});
fetch('https://example.com', {signal: controller.signal})
.then(res => res.json())
.then(
data => {
useData(data);
},
err => {
if (err.name === 'AbortError') {
console.log('request was aborted');
}
}
)
.finally(() => {
clearTimeout(timeout);
});
```
See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js) for more examples.
See [test cases](https://github.com/node-fetch/node-fetch/blob/master/test/test.js) for more examples.
## API
@ -314,6 +408,7 @@ Perform an HTTP(S) fetch.
`url` should be an absolute url, such as `https://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected `Promise`.
<a id="fetch-options"></a>
### Options
The default values are shown after each option key.
@ -322,36 +417,37 @@ The default values are shown after each option key.
{
// These properties are part of the Fetch Standard
method: 'GET',
headers: {}, // request headers. format is the identical to that accepted by the Headers constructor (see below)
body: null, // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream
redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect
signal: null, // pass an instance of AbortSignal to optionally abort requests
headers: {}, // request headers. format is the identical to that accepted by the Headers constructor (see below)
body: null, // request body. can be null, a string, a Buffer, a Blob, or a Node.js Readable stream
redirect: 'follow', // set to `manual` to extract redirect headers, `error` to reject redirect
signal: null, // pass an instance of AbortSignal to optionally abort requests
// The following properties are node-fetch extensions
follow: 20, // maximum redirect count. 0 to not follow redirect
timeout: 0, // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies). Signal is recommended instead.
compress: true, // support gzip/deflate content encoding. false to disable
size: 0, // maximum response body size in bytes. 0 to disable
agent: null // http(s).Agent instance or function that returns an instance (see below)
follow: 20, // maximum redirect count. 0 to not follow redirect
timeout: 0, // req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies). Signal is recommended instead.
compress: true, // support gzip/deflate content encoding. false to disable
size: 0, // maximum response body size in bytes. 0 to disable
agent: null, // http(s).Agent instance or function that returns an instance (see below)
highWaterMark: 16384 // the maximum number of bytes to store in the internal buffer before ceasing to read from the underlying resource.
}
```
##### Default Headers
#### Default Headers
If no values are set, the following request headers will be sent automatically:
Header | Value
------------------- | --------------------------------------------------------
`Accept-Encoding` | `gzip,deflate` _(when `options.compress === true`)_
`Accept` | `*/*`
`Connection` | `close` _(when no `options.agent` is present)_
`Content-Length` | _(automatically calculated, if possible)_
`Transfer-Encoding` | `chunked` _(when `req.body` is a stream)_
`User-Agent` | `node-fetch/1.0 (+https://github.com/bitinn/node-fetch)`
| Header | Value |
| ------------------- | -------------------------------------------------------- |
| `Accept-Encoding` | `gzip,deflate` _(when `options.compress === true`)_ |
| `Accept` | `*/*` |
| `Connection` | `close` _(when no `options.agent` is present)_ |
| `Content-Length` | _(automatically calculated, if possible)_ |
| `Transfer-Encoding` | `chunked` _(when `req.body` is a stream)_ |
| `User-Agent` | `node-fetch (+https://github.com/node-fetch/node-fetch)` |
Note: when `body` is a `Stream`, `Content-Length` is not set automatically.
##### Custom Agent
#### Custom Agent
The `agent` option allows you to specify networking related options which are out of the scope of Fetch, including and not limited to the following:
@ -364,25 +460,58 @@ See [`http.Agent`](https://nodejs.org/api/http.html#http_new_agent_options) for
In addition, the `agent` option accepts a function that returns `http`(s)`.Agent` instance given current [URL](https://nodejs.org/api/url.html), this is useful during a redirection chain across HTTP and HTTPS protocol.
```js
const http = require('http');
const https = require('https');
const httpAgent = new http.Agent({
keepAlive: true
keepAlive: true
});
const httpsAgent = new https.Agent({
keepAlive: true
keepAlive: true
});
const options = {
agent: function (_parsedURL) {
if (_parsedURL.protocol == 'http:') {
return httpAgent;
} else {
return httpsAgent;
}
}
}
agent: function(_parsedURL) {
if (_parsedURL.protocol == 'http:') {
return httpAgent;
} else {
return httpsAgent;
}
}
};
```
<a id="custom-highWaterMark"></a>
#### Custom highWaterMark
Stream on Node.js have a smaller internal buffer size (16Kb, aka `highWaterMark`) from client-side browsers (>1Mb, not consistent across browsers). Because of that, when you are writing an isomorphic app and using `res.clone()`, it will hang with large response in Node.
The recommended way to fix this problem is to resolve cloned response in parallel:
```js
const fetch = require('node-fetch');
fetch('https://example.com').then(res => {
const r1 = res.clone();
return Promise.all([res.json(), r1.text()]).then(results => {
console.log(results[0]);
console.log(results[1]);
});
});
```
If for some reason you don't like the solution above, since `3.x` you are able to modify the `highWaterMark` option:
```js
const fetch = require('node-fetch');
fetch('https://example.com', {highWaterMark: 10}).then(res => res.clone().buffer());
```
<a id="class-request"></a>
### Class: Request
An HTTP(S) request containing information about URL, method, headers, and the body. This class implements the [Body](#iface-body) interface.
@ -405,12 +534,13 @@ The following node-fetch extension properties are provided:
- `compress`
- `counter`
- `agent`
- `highWaterMark`
See [options](#fetch-options) for exact meaning of these extensions.
#### new Request(input[, options])
<small>*(spec-compliant)*</small>
<small>_(spec-compliant)_</small>
- `input` A string representing a URL, or another `Request` (which will be cloned)
- `options` [Options][#fetch-options] for the HTTP(S) request
@ -420,6 +550,7 @@ Constructs a new `Request` object. The constructor is identical to that in the [
In most cases, directly `fetch(url, options)` is simpler than creating a `Request` object.
<a id="class-response"></a>
### Class: Response
An HTTP(S) response. This class implements the [Body](#iface-body) interface.
@ -433,7 +564,7 @@ The following properties are not implemented in node-fetch at this moment:
#### new Response([body[, options]])
<small>*(spec-compliant)*</small>
<small>_(spec-compliant)_</small>
- `body` A `String` or [`Readable` stream][node-readable]
- `options` A [`ResponseInit`][response-init] options dictionary
@ -444,24 +575,25 @@ Because Node.js does not implement service workers (for which this class was des
#### response.ok
<small>*(spec-compliant)*</small>
<small>_(spec-compliant)_</small>
Convenience property representing if the request ended normally. Will evaluate to true if the response status was greater than or equal to 200 but smaller than 300.
#### response.redirected
<small>*(spec-compliant)*</small>
<small>_(spec-compliant)_</small>
Convenience property representing if the request has been redirected at least once. Will evaluate to true if the internal redirect counter is greater than 0.
<a id="class-headers"></a>
### Class: Headers
This class allows manipulating and iterating over a set of HTTP headers. All methods specified in the [Fetch Standard][whatwg-fetch] are implemented.
#### new Headers([init])
<small>*(spec-compliant)*</small>
<small>_(spec-compliant)_</small>
- `init` Optional argument to pre-fill the `Headers` object
@ -469,18 +601,16 @@ Construct a new `Headers` object. `init` can be either `null`, a `Headers` objec
```js
// Example adapted from https://fetch.spec.whatwg.org/#example-headers-class
const Headers = require('node-fetch');
const meta = {
'Content-Type': 'text/xml',
'Breaking-Bad': '<3'
'Content-Type': 'text/xml',
'Breaking-Bad': '<3'
};
const headers = new Headers(meta);
// The above is equivalent to
const meta = [
[ 'Content-Type', 'text/xml' ],
[ 'Breaking-Bad', '<3' ]
];
const meta = [['Content-Type', 'text/xml'], ['Breaking-Bad', '<3']];
const headers = new Headers(meta);
// You can in fact use any iterable objects, like a Map or even another Headers
@ -492,6 +622,7 @@ const copyOfHeaders = new Headers(headers);
```
<a id="iface-body"></a>
### Interface: Body
`Body` is an abstract interface with methods that are applicable to both `Request` and `Response` classes.
@ -502,89 +633,89 @@ The following methods are not yet implemented in node-fetch at this moment:
#### body.body
<small>*(deviation from spec)*</small>
<small>_(deviation from spec)_</small>
* Node.js [`Readable` stream][node-readable]
- Node.js [`Readable` stream][node-readable]
Data are encapsulated in the `Body` object. Note that while the [Fetch Standard][whatwg-fetch] requires the property to always be a WHATWG `ReadableStream`, in node-fetch it is a Node.js [`Readable` stream][node-readable].
#### body.bodyUsed
<small>*(spec-compliant)*</small>
<small>_(spec-compliant)_</small>
* `Boolean`
- `Boolean`
A boolean property for if this body has been consumed. Per the specs, a consumed body cannot be used again.
#### body.arrayBuffer()
#### body.blob()
#### body.json()
#### body.text()
<small>*(spec-compliant)*</small>
<small>_(spec-compliant)_</small>
* Returns: <code>Promise</code>
- Returns: `Promise`
Consume the body and return a promise that will resolve to one of these formats.
#### body.buffer()
<small>*(node-fetch extension)*</small>
<small>_(node-fetch extension)_</small>
* Returns: <code>Promise&lt;Buffer&gt;</code>
- Returns: `Promise<Buffer>`
Consume the body and return a promise that will resolve to a Buffer.
#### body.textConverted()
<small>*(node-fetch extension)*</small>
* Returns: <code>Promise&lt;String&gt;</code>
Identical to `body.text()`, except instead of always converting to UTF-8, encoding sniffing will be performed and text converted to UTF-8 if possible.
(This API requires an optional dependency of the npm package [encoding](https://www.npmjs.com/package/encoding), which you need to install manually. `webpack` users may see [a warning message](https://github.com/bitinn/node-fetch/issues/412#issuecomment-379007792) due to this optional dependency.)
<a id="class-fetcherror"></a>
### Class: FetchError
<small>*(node-fetch extension)*</small>
<small>_(node-fetch extension)_</small>
An operational error in the fetching process. See [ERROR-HANDLING.md][] for more info.
<a id="class-aborterror"></a>
### Class: AbortError
<small>*(node-fetch extension)*</small>
<small>_(node-fetch extension)_</small>
An Error thrown when the request is aborted in response to an `AbortSignal`'s `abort` event. It has a `name` property of `AbortError`. See [ERROR-HANDLING.MD][] for more info.
## TypeScript
Since `3.x` types are bundled with `node-fetch`, so you don't need to install any additional packages.
For older versions please use the type definitions from [DefinitelyTyped](https://github.com/DefinitelyTyped/DefinitelyTyped):
```sh
$ npm install --save-dev @types/node-fetch
```
## Acknowledgement
Thanks to [github/fetch](https://github.com/github/fetch) for providing a solid implementation reference.
`node-fetch` v1 was maintained by [@bitinn](https://github.com/bitinn); v2 was maintained by [@TimothyGu](https://github.com/timothygu), [@bitinn](https://github.com/bitinn) and [@jimmywarting](https://github.com/jimmywarting); v2 readme is written by [@jkantr](https://github.com/jkantr).
## Team
[![David Frank](https://github.com/bitinn.png?size=100)](https://github.com/bitinn) | [![Jimmy Wärting](https://github.com/jimmywarting.png?size=100)](https://github.com/jimmywarting) | [![Antoni Kepinski](https://github.com/xxczaki.png?size=100)](https://github.com/xxczaki) | [![Richie Bendall](https://github.com/Richienb.png?size=100)](https://github.com/Richienb) | [![Gregor Martynus](https://github.com/gr2m.png?size=100)](https://github.com/gr2m)
---|---|---|---|---
[David Frank](https://bitinn.net/) | [Jimmy Wärting](https://jimmy.warting.se/) | [Antoni Kepinski](https://kepinski.me) | [Richie Bendall](https://www.richie-bendall.ml/) | [Gregor Martynus](https://twitter.com/gr2m)
###### Former
- [Timothy Gu](https://github.com/timothygu)
- [Jared Kantrowitz](https://github.com/jkantr)
## License
MIT
[npm-image]: https://flat.badgen.net/npm/v/node-fetch
[npm-url]: https://www.npmjs.com/package/node-fetch
[travis-image]: https://flat.badgen.net/travis/bitinn/node-fetch
[travis-url]: https://travis-ci.org/bitinn/node-fetch
[codecov-image]: https://img.shields.io/codecov/c/gh/node-fetch/node-fetch/master?style=flat-square
[codecov-url]: https://codecov.io/gh/node-fetch/node-fetch
[install-size-image]: https://flat.badgen.net/packagephobia/install/node-fetch
[install-size-url]: https://packagephobia.now.sh/result?p=node-fetch
[discord-image]: https://img.shields.io/discord/619915844268326952?color=%237289DA&label=Discord&style=flat-square
[discord-url]: https://discord.gg/Zxbndcm
[opencollective-image]: https://opencollective.com/node-fetch/backers.svg
[opencollective-url]: https://opencollective.com/node-fetch
[whatwg-fetch]: https://fetch.spec.whatwg.org/
[response-init]: https://fetch.spec.whatwg.org/#responseinit
[node-readable]: https://nodejs.org/api/stream.html#stream_readable_streams
[mdn-headers]: https://developer.mozilla.org/en-US/docs/Web/API/Headers
[LIMITS.md]: https://github.com/bitinn/node-fetch/blob/master/LIMITS.md
[ERROR-HANDLING.md]: https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md
[UPGRADE-GUIDE.md]: https://github.com/bitinn/node-fetch/blob/master/UPGRADE-GUIDE.md
[error-handling.md]: https://github.com/node-fetch/node-fetch/blob/master/docs/ERROR-HANDLING.md

View File

@ -1,25 +0,0 @@
"use strict";
// ref: https://github.com/tc39/proposal-global
var getGlobal = function () {
// the only reliable means to get the global object is
// `Function('return this')()`
// However, this causes CSP violations in Chrome apps.
if (typeof self !== 'undefined') { return self; }
if (typeof window !== 'undefined') { return window; }
if (typeof global !== 'undefined') { return global; }
throw new Error('unable to locate global object');
}
var global = getGlobal();
module.exports = exports = global.fetch;
// Needed for TypeScript and Webpack.
if (global.fetch) {
exports.default = global.fetch.bind(global);
}
exports.Headers = global.Headers;
exports.Request = global.Request;
exports.Response = global.Response;

View File

@ -1,61 +0,0 @@
// This Babel plugin makes it possible to do CommonJS-style function exports
const walked = Symbol('walked');
module.exports = ({ types: t }) => ({
visitor: {
Program: {
exit(program) {
if (program[walked]) {
return;
}
for (let path of program.get('body')) {
if (path.isExpressionStatement()) {
const expr = path.get('expression');
if (expr.isAssignmentExpression() &&
expr.get('left').matchesPattern('exports.*')) {
const prop = expr.get('left').get('property');
if (prop.isIdentifier({ name: 'default' })) {
program.unshiftContainer('body', [
t.expressionStatement(
t.assignmentExpression('=',
t.identifier('exports'),
t.assignmentExpression('=',
t.memberExpression(
t.identifier('module'), t.identifier('exports')
),
expr.node.right
)
)
),
t.expressionStatement(
t.callExpression(
t.memberExpression(
t.identifier('Object'), t.identifier('defineProperty')),
[
t.identifier('exports'),
t.stringLiteral('__esModule'),
t.objectExpression([
t.objectProperty(t.identifier('value'), t.booleanLiteral(true))
])
]
)
),
t.expressionStatement(
t.assignmentExpression('=',
expr.node.left, t.identifier('exports')
)
)
]);
path.remove();
}
}
}
}
program[walked] = true;
}
}
}
});

View File

@ -1,18 +0,0 @@
export default function tweakDefault() {
return {
transformBundle: function (source) {
var lines = source.split('\n');
for (var i = 0; i < lines.length; i++) {
var line = lines[i];
var matches = /^(exports(?:\['default']|\.default)) = (.*);$/.exec(line);
if (matches) {
lines[i] = 'module.exports = exports = ' + matches[2] + ';\n' +
'Object.defineProperty(exports, "__esModule", { value: true });\n' +
matches[1] + ' = exports;';
break;
}
}
return lines.join('\n');
}
};
}

View File

@ -1,7 +1,32 @@
Changelog
=========
# 3.x release
## v3.0.0
<!--- Not completed yet, since the v3 roadmap is not fully finished -->
- **Breaking:** minimum supported Node.js version is now 10.
- Enhance: added new node-fetch-only option: `highWaterMark`.
- Enhance: `AbortError` now uses a w3c defined message.
- Enhance: data URI support.
- Enhance: drop existing blob implementation code and use fetch-blob as dependency instead.
- Enhance: modernise the code behind `FetchError` and `AbortError`.
- Enhance: replace deprecated `url.parse()` and `url.replace()` with the new WHATWG's `new URL()`
- Enhance: allow excluding a `user-agent` in a fetch request by setting it's header to null.
- Fix: `Response.statusText` no longer sets a default message derived from the HTTP status code.
- Fix: missing response stream error events.
- Fix: do not use constructor.name to check object.
- Fix: convert `Content-Encoding` to lowercase.
- Fix: propagate size and timeout to cloned response.
- Other: bundle TypeScript types.
- Other: replace Rollup with @pika/pack.
- Other: introduce linting to the project.
- Other: simplify Travis CI build matrix.
- Other: dev dependency update.
- Other: readme update.
# 2.x release
@ -40,7 +65,7 @@ Changelog
## v2.2.1
- Fix: `compress` flag shouldn't overwrite existing `Accept-Encoding` header.
- Fix: multiple `import` rules, where `PassThrough` etc. doesn't have a named export when using node <10 and `--exerimental-modules` flag.
- Fix: multiple `import` rules, where `PassThrough` etc. doesn't have a named export when using node <10 and `--experimental-modules` flag.
- Other: Better README.
## v2.2.0
@ -74,7 +99,7 @@ Fix packaging errors in v2.1.0.
## v2.0.0
This is a major release. Check [our upgrade guide](https://github.com/bitinn/node-fetch/blob/master/UPGRADE-GUIDE.md) for an overview on some key differences between v1 and v2.
This is a major release. Check [our upgrade guide](https://github.com/node-fetch/node-fetch/blob/master/UPGRADE-GUIDE.md) for an overview on some key differences between v1 and v2.
### General changes
@ -99,7 +124,7 @@ This is a major release. Check [our upgrade guide](https://github.com/bitinn/nod
### Response and Request classes
- Major: `response.text()` no longer attempts to detect encoding, instead always opting for UTF-8 (per spec); use `response.textConverted()` for the v1 behavior
- Major: make `response.json()` throw error instead of returning an empty object on 204 no-content respose (per spec; reverts behavior changed in v1.6.2)
- Major: make `response.json()` throw error instead of returning an empty object on 204 no-content response (per spec; reverts behavior changed in v1.6.2)
- Major: internal methods are no longer exposed
- Major: throw error when a `GET` or `HEAD` Request is constructed with a non-null body (per spec)
- Enhance: add `response.arrayBuffer()` (also applies to Requests)
@ -124,9 +149,9 @@ This is a major release. Check [our upgrade guide](https://github.com/bitinn/nod
# 1.x release
## backport releases (v1.7.0 and beyond)
## Backport releases (v1.7.0 and beyond)
See [changelog on 1.x branch](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) for details.
See [changelog on 1.x branch](https://github.com/node-fetch/node-fetch/blob/1.x/CHANGELOG.md) for details.
## v1.6.3

View File

@ -6,17 +6,19 @@ Because `window.fetch` isn't designed to be transparent about the cause of reque
The basics:
- A cancelled request is rejected with an [`AbortError`](https://github.com/bitinn/node-fetch/blob/master/README.md#class-aborterror). You can check if the reason for rejection was that the request was aborted by checking the `Error`'s `name` is `AbortError`.
- A cancelled request is rejected with an [`AbortError`](https://github.com/node-fetch/node-fetch/blob/master/README.md#class-aborterror). You can check if the reason for rejection was that the request was aborted by checking the `Error`'s `name` is `AbortError`.
```js
fetch(url, { signal }).catch(err => {
if (err.name === 'AbortError') {
// request was aborted
const fetch = required('node-fetch');
fetch(url, {signal}).catch(error => {
if (error.name === 'AbortError') {
console.log('request was aborted');
}
})
});
```
- All [operational errors][joyent-guide] *other than aborted requests* are rejected with a [FetchError](https://github.com/bitinn/node-fetch/blob/master/README.md#class-fetcherror). You can handle them all through the promise `catch` clause.
- All [operational errors][joyent-guide] *other than aborted requests* are rejected with a [FetchError](https://github.com/node-fetch/node-fetch/blob/master/README.md#class-fetcherror). You can handle them all through the promise `catch` clause.
- All errors come with an `err.message` detailing the cause of errors.
@ -28,6 +30,6 @@ fetch(url, { signal }).catch(err => {
List of error types:
- Because we maintain 100% coverage, see [test.js](https://github.com/bitinn/node-fetch/blob/master/test/test.js) for a full list of custom `FetchError` types, as well as some of the common errors from Node.js
- Because we maintain 100% coverage, see [test.js](https://github.com/node-fetch/node-fetch/blob/master/test/test.js) for a full list of custom `FetchError` types, as well as some of the common errors from Node.js
[joyent-guide]: https://www.joyent.com/node-js/production/design/errors#operational-errors-vs-programmer-errors

21
docs/media/Banner.svg Normal file
View File

@ -0,0 +1,21 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="1280px" height="720px" viewBox="0 0 1280 720" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg">
<desc>Created with Lunacy</desc>
<defs>
<rect width="1280" height="720" id="artboard_1" />
<clipPath id="clip_1">
<use xlink:href="#artboard_1" clip-rule="evenodd" />
</clipPath>
</defs>
<g id="Banner" clip-path="url(#clip_1)">
<use xlink:href="#artboard_1" stroke="none" fill="#FFFFFF" />
<g id="Headline-2" transform="translate(116 252)">
<g id="Node">
<path d="M294.95 180.32C307.95 180.32 316.95 175.12 321.75 166.12L321.15 178.52L348.15 178.52L348.15 34.1195L321.15 34.1195L321.15 105.32C315.75 96.9195 308.15 91.5195 294.55 91.5195C273.95 91.5195 256.35 109.52 256.35 135.72C256.35 161.92 274.15 180.32 294.95 180.32ZM128.35 179.72L125.55 179.72L56.95 115.32C50.15 108.92 40.75 96.7195 40.75 96.7195C40.75 96.7195 42.75 111.72 42.75 121.52L42.75 178.52L16.55 178.52L16.55 41.3195L19.35 41.3195L87.95 105.72C94.55 111.92 103.95 124.32 103.95 124.32C103.95 124.32 102.15 108.92 102.15 99.5195L102.15 42.5195L128.35 42.5195L128.35 179.72ZM242.45 136.12C242.45 161.92 222.05 180.32 195.65 180.32C169.25 180.32 148.65 161.92 148.65 136.12C148.65 110.12 169.25 91.5195 195.65 91.5195C222.05 91.5195 242.45 110.12 242.45 136.12ZM457.05 144.12L392.45 144.12C395.05 152.92 402.25 157.92 414.45 157.92C424.25 157.92 431.65 154.12 435.85 151.12L451.25 167.72C443.05 174.92 432.65 180.32 414.85 180.32C384.45 180.32 364.85 161.92 364.85 135.72C364.85 109.92 385.45 91.5195 411.85 91.5195C442.45 91.5195 459.05 114.32 457.05 144.12ZM412.05 113.52C401.45 113.52 394.25 118.32 392.05 128.52L429.85 128.52C428.65 119.12 422.65 113.52 412.05 113.52ZM176.25 136.12C176.25 148.72 183.65 157.52 195.65 157.52C207.65 157.52 214.85 148.72 214.85 136.12C214.85 123.32 207.65 114.72 195.65 114.72C183.65 114.72 176.25 123.32 176.25 136.12ZM302.75 156.72C291.35 156.72 284.15 149.12 284.15 135.72C284.15 123.12 291.35 115.32 302.75 115.32C314.35 115.32 321.95 123.12 321.95 135.72C321.95 149.12 314.35 156.72 302.75 156.72Z" />
</g>
<g id="Fetch" fill="#4CAF50">
<path d="M929.75 178.52L902.75 178.52L902.75 34.1195L929.75 34.1195L929.75 106.72C934.95 96.9195 944.35 91.5195 956.15 91.5195C974.75 91.5195 988.15 105.12 988.15 129.32L988.15 178.52L961.15 178.52L961.15 131.12C961.15 121.52 956.55 116.12 946.95 116.12C936.15 116.12 929.75 122.12 929.75 132.92L929.75 178.52ZM521.75 178.52L549.75 178.52L549.75 137.12L603.95 137.12L603.95 112.12L549.75 112.12L549.75 67.1195L609.35 67.1195L609.35 42.5195L521.75 42.5195L521.75 178.52ZM795.55 172.92C791.35 176.32 784.15 180.72 771.75 180.72C754.55 180.72 742.15 172.12 742.15 146.52L742.15 115.72L729.55 115.72L729.55 93.3195L742.15 93.3195L742.15 48.9195L768.95 48.9195L768.95 93.3195L790.75 93.3195L790.75 115.72L768.95 115.72L768.95 147.32C768.95 155.92 771.75 157.72 775.95 157.72C780.35 157.72 783.55 155.32 785.35 154.12L795.55 172.92ZM652.45 144.12L717.05 144.12C719.05 114.32 702.45 91.5195 671.85 91.5195C645.45 91.5195 624.85 109.92 624.85 135.72C624.85 161.92 644.45 180.32 674.85 180.32C692.65 180.32 703.05 174.92 711.25 167.72L695.85 151.12C691.65 154.12 684.25 157.92 674.45 157.92C662.25 157.92 655.05 152.92 652.45 144.12ZM852.45 180.32C869.45 180.32 879.45 174.32 887.85 164.92L869.65 148.92C865.65 152.92 861.05 156.92 852.45 156.92C840.25 156.92 833.45 148.12 833.45 135.72C833.45 123.72 840.25 115.12 852.45 115.12C859.25 115.12 865.85 118.92 869.05 124.12L888.25 108.52C879.65 97.7195 869.45 91.5195 852.45 91.5195C825.85 91.5195 805.85 109.92 805.85 135.72C805.85 161.92 825.85 180.32 852.45 180.32ZM652.05 128.52C654.25 118.32 661.45 113.52 672.05 113.52C682.65 113.52 688.65 119.12 689.85 128.52L652.05 128.52Z" />
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 3.8 KiB

21
docs/media/Logo.svg Normal file
View File

@ -0,0 +1,21 @@
<?xml version="1.0" encoding="utf-8"?>
<svg width="512px" height="512px" viewBox="0 0 512 512" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg">
<desc>Created with Lunacy</desc>
<defs>
<rect width="512" height="512" id="artboard_1" />
<clipPath id="clip_1">
<use xlink:href="#artboard_1" clip-rule="evenodd" />
</clipPath>
</defs>
<g id="Logo" clip-path="url(#clip_1)">
<use xlink:href="#artboard_1" stroke="none" fill="#FFFFFF" />
<g id="Headline-1" transform="translate(1 0)">
<g id="N" fill="#000000" fill-opacity="0.870588243">
<path d="M251.625 380.634L256.525 380.634L256.525 140.534L210.675 140.534L210.675 240.284C210.675 256.734 213.825 283.684 213.825 283.684C213.825 283.684 197.375 261.984 185.825 251.134L65.775 138.434L60.875 138.434L60.875 378.534L106.725 378.534L106.725 278.784C106.725 261.634 103.225 235.384 103.225 235.384C103.225 235.384 119.675 256.734 131.575 267.934L251.625 380.634Z" />
</g>
<g id="F" fill="#4CAF50" fill-opacity="0.870588243">
<path d="M306.225 378.534L355.225 378.534L355.225 306.084L450.075 306.084L450.075 262.334L355.225 262.334L355.225 183.584L459.525 183.584L459.525 140.534L306.225 140.534L306.225 378.534Z" />
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

BIN
docs/media/NodeFetch.sketch Normal file

Binary file not shown.

View File

@ -26,7 +26,7 @@ Known differences
- If you are using `res.clone()` and writing an isomorphic app, note that stream on Node.js have a smaller internal buffer size (16Kb, aka `highWaterMark`) from client-side browsers (>1Mb, not consistent across browsers).
- Because node.js stream doesn't expose a [*disturbed*](https://fetch.spec.whatwg.org/#concept-readablestream-disturbed) property like Stream spec, using a consumed stream for `new Response(body)` will not set `bodyUsed` flag correctly.
- Because Node.js stream doesn't expose a [*disturbed*](https://fetch.spec.whatwg.org/#concept-readablestream-disturbed) property like Stream spec, using a consumed stream for `new Response(body)` will not set `bodyUsed` flag correctly.
[readable-stream]: https://nodejs.org/api/stream.html#stream_readable_streams
[ERROR-HANDLING.md]: https://github.com/bitinn/node-fetch/blob/master/ERROR-HANDLING.md
[ERROR-HANDLING.md]: https://github.com/node-fetch/node-fetch/blob/master/docs/ERROR-HANDLING.md

View File

@ -45,7 +45,7 @@ spec-compliant. These changes are done in conjunction with GitHub's
const headers = new Headers({
'Abc': 'string',
'Multi': [ 'header1', 'header2' ]
'Multi': ['header1', 'header2']
});
// before after
@ -63,14 +63,14 @@ headers.get('Multi') => headers.get('Multi') =>
const headers = new Headers({
'Abc': 'string',
'Multi': [ 'header1', 'header2' ]
'Multi': ['header1', 'header2']
});
// before after
headers.getAll('Multi') => headers.getAll('Multi') =>
[ 'header1', 'header2' ]; throws ReferenceError
headers.get('Multi').split(',') =>
[ 'header1', 'header2' ];
['header1', 'header2'];
//////////////////////////////////////////////////////////////////////////////
@ -91,7 +91,7 @@ headers.get(undefined) headers.get(undefined)
const headers = new Headers();
headers.set('Héy', 'ok'); // now throws
headers.get('Héy'); // now throws
new Headers({ 'Héy': 'ok' }); // now throws
new Headers({'Héy': 'ok'}); // now throws
```
## Node.js v0.x support dropped

31
docs/v3-LIMITS.md Normal file
View File

@ -0,0 +1,31 @@
Known differences
=================
*As of 3.x release*
- Topics such as Cross-Origin, Content Security Policy, Mixed Content, Service Workers are ignored, given our server-side context.
- On the upside, there are no forbidden headers.
- `res.url` contains the final url when following redirects.
- For convenience, `res.body` is a Node.js [Readable stream][readable-stream], so decoding can be handled independently.
- Similarly, `req.body` can either be `null`, a buffer or a Readable stream.
- Also, you can handle rejected fetch requests through checking `err.type` and `err.code`. See [ERROR-HANDLING.md][] for more info.
- Only support `res.text()`, `res.json()`, `res.blob()`, `res.arraybuffer()`, `res.buffer()`
- There is currently no built-in caching, as server-side caching varies by use-cases.
- Current implementation lacks server-side cookie store, you will need to extract `Set-Cookie` headers manually.
- If you are using `res.clone()` and writing an isomorphic app, note that stream on Node.js have a smaller internal buffer size (16Kb, aka `highWaterMark`) from client-side browsers (>1Mb, not consistent across browsers). Learn [how to get around this][highwatermark-fix].
- Because Node.js stream doesn't expose a [*disturbed*](https://fetch.spec.whatwg.org/#concept-readablestream-disturbed) property like Stream spec, using a consumed stream for `new Response(body)` will not set `bodyUsed` flag correctly.
[readable-stream]: https://nodejs.org/api/stream.html#stream_readable_streams
[ERROR-HANDLING.md]: https://github.com/node-fetch/node-fetch/blob/master/docs/ERROR-HANDLING.md
[highwatermark-fix]: https://github.com/node-fetch/node-fetch/blob/master/README.md#custom-highwatermark

110
docs/v3-UPGRADE-GUIDE.md Normal file
View File

@ -0,0 +1,110 @@
# Upgrade to node-fetch v3.x
node-fetch v3.x brings about many changes that increase the compliance of
WHATWG's [Fetch Standard][whatwg-fetch]. However, many of these changes mean
that apps written for node-fetch v2.x needs to be updated to work with
node-fetch v3.x and be conformant with the Fetch Standard. This document helps
you make this transition.
Note that this document is not an exhaustive list of all changes made in v3.x,
but rather that of the most important breaking changes. See our [changelog] for
other comparatively minor modifications.
- [Breaking Changes](#breaking)
- [Enhancements](#enhancements)
---
<a id="breaking"></a>
# Breaking Changes
## Minimum supported Node.js version is now 10
Since Node.js will deprecate version 8 at the end of 2019, we decided that node-fetch v3.x will not only drop support for Node.js 4 and 6 (which were supported in v2.x), but also for Node.js 8. We strongly encourage you to upgrade, if you still haven't done so. Check out Node.js' official [LTS plan] for more information on Node.js' support lifetime.
## `Response.statusText` no longer sets a default message derived from the HTTP status code
If the server didn't respond with status text, node-fetch would set a default message derived from the HTTP status code. This behavior was not spec-compliant and now the `statusText` will remain blank instead.
## Dropped the `browser` field in package.json
Prior to v3.x, we included a `browser` field in the package.json file. Since node-fetch is intended to be used on the server, we have removed this field. If you are using node-fetch client-side, consider switching to something like [cross-fetch].
## Dropped the `res.textConverted()` function
If you want charset encoding detection, please use the [fetch-charset-detection] package ([documentation][fetch-charset-detection-docs]).
```js
const fetch = require("node-fetch");
const convertBody = require("fetch-charset-detection");
fetch("https://somewebsite.com").then(res => {
const text = convertBody(res.buffer(), res.headers);
});
```
## JSON parsing errors from `res.json()` are of type `SyntaxError` instead of `FetchError`
When attempting to parse invalid json via `res.json()`, a `SyntaxError` will now be thrown instead of a `FetchError` to align better with the spec.
```js
const fetch = require("node-fetch");
fetch("https://somewebsitereturninginvalidjson.com").then(res => res.json())
// Throws 'Uncaught SyntaxError: Unexpected end of JSON input' or similar.
```
## A stream pipeline is now used to forward errors
If you are listening for errors via `res.body.on('error', () => ...)`, replace it with `res.body.once('error', () => ...)` so that your callback is not [fired twice](https://github.com/node-fetch/node-fetch/issues/668#issuecomment-569386115) in NodeJS >=13.5.
## `req.body` can no longer be a string
We are working towards changing body to become either null or a stream.
## Changed default user agent
The default user agent has been changed from `node-fetch/1.0 (+https://github.com/node-fetch/node-fetch)` to `node-fetch (+https://github.com/node-fetch/node-fetch)`.
## Arbitrary URLs are no longer supported
Since in 3.x we are using the WHATWG's `new URL()`, arbitrary URL parsing will fail due to lack of base.
# Enhancements
## Data URI support
Previously, node-fetch only supported http url scheme. However, the Fetch Standard recently introduced the `data:` URI support. Following the specification, we implemented this feature in v3.x. Read more about `data:` URLs [here][data-url].
## New & exposed Blob implementation
Blob implementation is now [fetch-blob] and hence is exposed, unlikely previously, where Blob type was only internal and not exported.
## Better UTF-8 URL handling
We now use the new Node.js [WHATWG-compliant URL API][whatwg-nodejs-url], so UTF-8 URLs are handled properly.
## Request errors are now piped using `stream.pipeline`
Since the v3.x required at least Node.js 10, we can utilise the new API.
## Creating Request/Response objects with relative URLs is no longer supported
We introduced Node.js `new URL()` API in 3.x, because it offers better UTF-8 support and is WHATWG URL compatible. The drawback is, given current limit of the API (nodejs/node#12682), it's not possible to support relative URL parsing without hacks.
Due to the lack of a browsing context in Node.js, we opted to drop support for relative URLs on Request/Response object, and it will now throw errors if you do so.
The main `fetch()` function will support absolute URLs and data url.
## Bundled TypeScript types
Since v3.x you no longer need to install `@types/node-fetch` package in order to use `node-fetch` with TypeScript.
[whatwg-fetch]: https://fetch.spec.whatwg.org/
[data-url]: https://fetch.spec.whatwg.org/#data-url-processor
[LTS plan]: https://github.com/nodejs/LTS#lts-plan
[cross-fetch]: https://github.com/lquixada/cross-fetch
[fetch-charset-detection]: https://github.com/Richienb/fetch-charset-detection
[fetch-charset-detection-docs]: https://richienb.github.io/fetch-charset-detection/globals.html#convertbody
[fetch-blob]: https://github.com/bitinn/fetch-blob#readme
[whatwg-nodejs-url]: https://nodejs.org/api/url.html#url_the_whatwg_url_api
[changelog]: CHANGELOG.md

27
example.js Normal file
View File

@ -0,0 +1,27 @@
const fetch = require('node-fetch');
// Plain text or HTML
fetch('https://github.com/')
.then(res => res.text())
.then(body => console.log(body));
// JSON
fetch('https://api.github.com/users/github')
.then(res => res.json())
.then(json => console.log(json));
// Simple Post
fetch('https://httpbin.org/post', {method: 'POST', body: 'a=1'})
.then(res => res.json())
.then(json => console.log(json));
// Post with JSON
const body = {a: 1};
fetch('https://httpbin.org/post', {
method: 'post',
body: JSON.stringify(body),
headers: {'Content-Type': 'application/json'}
})
.then(res => res.json())
.then(json => console.log(json));

21
externals.d.ts vendored Normal file
View File

@ -0,0 +1,21 @@
// `AbortSignal` is defined here to prevent a dependency on a particular
// implementation like the `abort-controller` package, and to avoid requiring
// the `dom` library in `tsconfig.json`.
export interface AbortSignal {
aborted: boolean;
addEventListener: (type: 'abort', listener: ((this: AbortSignal, event: any) => any), options?: boolean | {
capture?: boolean;
once?: boolean;
passive?: boolean;
}) => void;
removeEventListener: (type: 'abort', listener: ((this: AbortSignal, event: any) => any), options?: boolean | {
capture?: boolean;
}) => void;
dispatchEvent: (event: any) => boolean;
onabort?: null | ((this: AbortSignal, event: any) => void);
}

220
index.d.ts vendored Normal file
View File

@ -0,0 +1,220 @@
// Prior contributors: Torsten Werner <https://github.com/torstenwerner>
// Niklas Lindgren <https://github.com/nikcorg>
// Vinay Bedre <https://github.com/vinaybedre>
// Antonio Román <https://github.com/kyranet>
// Andrew Leedham <https://github.com/AndrewLeedham>
// Jason Li <https://github.com/JasonLi914>
// Brandon Wilson <https://github.com/wilsonianb>
// Steve Faulkner <https://github.com/southpolesteve>
/// <reference types="node" />
import {Agent} from 'http';
import {AbortSignal} from '../externals';
export class Request extends Body {
method: string;
redirect: RequestRedirect;
referrer: string;
url: string;
// Node-fetch extensions to the whatwg/fetch spec
agent?: Agent | ((parsedUrl: URL) => Agent);
compress: boolean;
counter: number;
follow: number;
hostname: string;
port?: number;
protocol: string;
size: number;
timeout: number;
highWaterMark?: number;
context: RequestContext;
headers: Headers;
constructor(input: string | { href: string } | Request, init?: RequestInit);
static redirect(url: string, status?: number): Response;
clone(): Request;
}
export interface RequestInit {
// Whatwg/fetch standard options
body?: BodyInit;
headers?: HeadersInit;
method?: string;
redirect?: RequestRedirect;
signal?: AbortSignal | null;
// Node-fetch extensions
agent?: Agent | ((parsedUrl: URL) => Agent); // =null http.Agent instance, allows custom proxy, certificate etc.
compress?: boolean; // =true support gzip/deflate content encoding. false to disable
follow?: number; // =20 maximum redirect count. 0 to not follow redirect
size?: number; // =0 maximum response body size in bytes. 0 to disable
timeout?: number; // =0 req/res timeout in ms, it resets on redirect. 0 to disable (OS limit applies)
highWaterMark?: number; // =16384 the maximum number of bytes to store in the internal buffer before ceasing to read from the underlying resource.
// node-fetch does not support mode, cache or credentials options
}
export type RequestContext =
'audio'
| 'beacon'
| 'cspreport'
| 'download'
| 'embed'
| 'eventsource'
| 'favicon'
| 'fetch'
| 'font'
| 'form'
| 'frame'
| 'hyperlink'
| 'iframe'
| 'image'
| 'imageset'
| 'import'
| 'internal'
| 'location'
| 'manifest'
| 'object'
| 'ping'
| 'plugin'
| 'prefetch'
| 'script'
| 'serviceworker'
| 'sharedworker'
| 'style'
| 'subresource'
| 'track'
| 'video'
| 'worker'
| 'xmlhttprequest'
| 'xslt';
export type RequestMode = 'cors' | 'no-cors' | 'same-origin';
export type RequestRedirect = 'error' | 'follow' | 'manual';
export type RequestCredentials = 'omit' | 'include' | 'same-origin';
export type RequestCache =
'default'
| 'force-cache'
| 'no-cache'
| 'no-store'
| 'only-if-cached'
| 'reload';
export class Headers implements Iterable<[string, string]> {
constructor(init?: HeadersInit);
forEach(callback: (value: string, name: string) => void): void;
append(name: string, value: string): void;
delete(name: string): void;
get(name: string): string | null;
getAll(name: string): string[];
has(name: string): boolean;
raw(): { [k: string]: string[] };
set(name: string, value: string): void;
// Iterator methods
entries(): Iterator<[string, string]>;
keys(): Iterator<string>;
values(): Iterator<[string]>;
[Symbol.iterator](): Iterator<[string, string]>;
}
type BlobPart = ArrayBuffer | ArrayBufferView | Blob | string;
interface BlobOptions {
type?: string;
endings?: 'transparent' | 'native';
}
export class Blob {
readonly type: string;
readonly size: number;
constructor(blobParts?: BlobPart[], options?: BlobOptions);
slice(start?: number, end?: number): Blob;
}
export class Body {
body: NodeJS.ReadableStream;
bodyUsed: boolean;
size: number;
timeout: number;
constructor(body?: any, opts?: { size?: number; timeout?: number });
arrayBuffer(): Promise<ArrayBuffer>;
blob(): Promise<Blob>;
buffer(): Promise<Buffer>;
json(): Promise<any>;
text(): Promise<string>;
}
export class FetchError extends Error {
name: 'FetchError';
[Symbol.toStringTag]: 'FetchError';
type: string;
code?: string;
errno?: string;
constructor(message: string, type: string, systemError?: object);
}
export class AbortError extends Error {
type: string;
message: string;
name: 'AbortError';
[Symbol.toStringTag]: 'AbortError';
constructor(message: string);
}
export class Response extends Body {
headers: Headers;
ok: boolean;
redirected: boolean;
status: number;
statusText: string;
type: ResponseType;
url: string;
size: number;
timeout: number;
constructor(body?: BodyInit, init?: ResponseInit);
static error(): Response;
static redirect(url: string, status: number): Response;
clone(): Response;
}
export type ResponseType =
'basic'
| 'cors'
| 'default'
| 'error'
| 'opaque'
| 'opaqueredirect';
export interface ResponseInit {
headers?: HeadersInit;
size?: number;
status?: number;
statusText?: string;
timeout?: number;
url?: string;
}
export type HeadersInit = Headers | string[][] | { [key: string]: string };
// HeaderInit is exported to support backwards compatibility. See PR #34382
export type HeaderInit = HeadersInit;
export type BodyInit =
ArrayBuffer
| ArrayBufferView
| NodeJS.ReadableStream
| string
| URLSearchParams;
export type RequestInfo = string | Request;
declare function fetch(
url: RequestInfo,
init?: RequestInit
): Promise<Response>;
declare namespace fetch {
function isRedirect(code: number): boolean;
}
export default fetch;

View File

@ -1,66 +1,152 @@
{
"name": "node-fetch",
"version": "2.6.0",
"description": "A light-weight module that brings window.fetch to node.js",
"main": "lib/index",
"browser": "./browser.js",
"module": "lib/index.mjs",
"files": [
"lib/index.js",
"lib/index.mjs",
"lib/index.es.js",
"browser.js"
],
"engines": {
"node": "4.x || >=6.0.0"
},
"scripts": {
"build": "cross-env BABEL_ENV=rollup rollup -c",
"prepare": "npm run build",
"test": "cross-env BABEL_ENV=test mocha --require babel-register --throw-deprecation test/test.js",
"report": "cross-env BABEL_ENV=coverage nyc --reporter lcov --reporter text mocha -R spec test/test.js",
"coverage": "cross-env BABEL_ENV=coverage nyc --reporter json --reporter text mocha -R spec test/test.js && codecov -f coverage/coverage-final.json"
},
"repository": {
"type": "git",
"url": "https://github.com/bitinn/node-fetch.git"
},
"keywords": [
"fetch",
"http",
"promise"
],
"author": "David Frank",
"license": "MIT",
"bugs": {
"url": "https://github.com/bitinn/node-fetch/issues"
},
"homepage": "https://github.com/bitinn/node-fetch",
"devDependencies": {
"@ungap/url-search-params": "^0.1.2",
"abort-controller": "^1.1.0",
"abortcontroller-polyfill": "^1.3.0",
"babel-core": "^6.26.3",
"babel-plugin-istanbul": "^4.1.6",
"babel-preset-env": "^1.6.1",
"babel-register": "^6.16.3",
"chai": "^3.5.0",
"chai-as-promised": "^7.1.1",
"chai-iterator": "^1.1.1",
"chai-string": "~1.3.0",
"codecov": "^3.3.0",
"cross-env": "^5.2.0",
"form-data": "^2.3.3",
"is-builtin-module": "^1.0.0",
"mocha": "^5.0.0",
"nyc": "11.9.0",
"parted": "^0.1.1",
"promise": "^8.0.3",
"resumer": "0.0.0",
"rollup": "^0.63.4",
"rollup-plugin-babel": "^3.0.7",
"string-to-arraybuffer": "^1.0.2",
"whatwg-url": "^5.0.0"
},
"dependencies": {}
"name": "node-fetch",
"version": "3.0.0-beta.1",
"description": "A light-weight module that brings window.fetch to node.js",
"main": "dist/index.js",
"module": "dist/index.mjs",
"types": "types/index.d.ts",
"files": [
"src/**/*",
"dist/**/*",
"types/**/*.d.ts"
],
"engines": {
"node": ">=10.0.0"
},
"scripts": {
"build": "pika-pack --out dist/",
"prepare": "npm run build",
"prepublishOnly": "npm run build",
"test": "cross-env BABEL_ENV=test mocha --require @babel/register --throw-deprecation test/*.js",
"report": "cross-env BABEL_ENV=coverage nyc --reporter lcov --reporter text mocha -R spec test/*.js",
"coverage": "cross-env BABEL_ENV=coverage nyc --reporter json --reporter text mocha -R spec test/*.js && codecov -f coverage/coverage-final.json",
"lint": "xo"
},
"repository": {
"type": "git",
"url": "https://github.com/node-fetch/node-fetch.git"
},
"keywords": [
"fetch",
"http",
"promise"
],
"author": "David Frank",
"license": "MIT",
"bugs": {
"url": "https://github.com/node-fetch/node-fetch/issues"
},
"homepage": "https://github.com/node-fetch/node-fetch",
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/node-fetch"
},
"devDependencies": {
"@babel/core": "^7.8.7",
"@babel/preset-env": "^7.8.7",
"@babel/register": "^7.8.6",
"@pika/pack": "^0.5.0",
"@pika/plugin-build-node": "^0.9.2",
"@pika/plugin-build-types": "^0.9.2",
"@pika/plugin-copy-assets": "^0.9.2",
"@pika/plugin-standard-pkg": "^0.9.2",
"abort-controller": "^3.0.0",
"abortcontroller-polyfill": "^1.4.0",
"chai": "^4.2.0",
"chai-as-promised": "^7.1.1",
"chai-iterator": "^3.0.2",
"chai-string": "^1.5.0",
"codecov": "^3.6.5",
"cross-env": "^7.0.2",
"form-data": "^3.0.0",
"mocha": "^7.1.0",
"nyc": "^15.0.0",
"parted": "^0.1.1",
"promise": "^8.1.0",
"resumer": "0.0.0",
"string-to-arraybuffer": "^1.0.2",
"xo": "^0.28.0"
},
"dependencies": {
"data-uri-to-buffer": "^3.0.0",
"fetch-blob": "^1.0.5"
},
"@pika/pack": {
"pipeline": [
[
"@pika/plugin-standard-pkg"
],
[
"@pika/plugin-build-node"
],
[
"@pika/plugin-build-types"
],
[
"@pika/plugin-copy-assets",
{
"files": [
"externals.d.ts"
]
}
]
]
},
"xo": {
"envs": [
"node",
"browser"
],
"rules": {
"complexity": 0,
"promise/prefer-await-to-then": 0,
"no-mixed-operators": 0,
"no-negated-condition": 0,
"unicorn/prevent-abbreviations": 0
},
"ignores": [
"dist"
],
"overrides": [
{
"files": "test/**/*.js",
"envs": [
"node",
"mocha"
],
"rules": {
"max-nested-callbacks": 0,
"no-unused-expressions": 0,
"new-cap": 0,
"guard-for-in": 0
}
},
{
"files": "example.js",
"rules": {
"import/no-extraneous-dependencies": 0
}
}
]
},
"babel": {
"presets": [
[
"@babel/preset-env",
{
"targets": {
"node": true
}
}
]
]
},
"nyc": {
"require": [
"@babel/register"
],
"sourceMap": false,
"instrument": false
},
"runkitExampleFilename": "example.js"
}

View File

@ -1,27 +0,0 @@
import isBuiltin from 'is-builtin-module';
import babel from 'rollup-plugin-babel';
import tweakDefault from './build/rollup-plugin';
process.env.BABEL_ENV = 'rollup';
export default {
input: 'src/index.js',
output: [
{ file: 'lib/index.js', format: 'cjs', exports: 'named' },
{ file: 'lib/index.es.js', format: 'es', exports: 'named', intro: 'process.emitWarning("The .es.js file is deprecated. Use .mjs instead.");' },
{ file: 'lib/index.mjs', format: 'es', exports: 'named' },
],
plugins: [
babel({
runtimeHelpers: true
}),
tweakDefault()
],
external: function (id) {
if (isBuiltin(id)) {
return true;
}
id = id.split('/').slice(0, id[0] === '@' ? 2 : 1).join('/');
return !!require('./package.json').dependencies[id];
}
};

View File

@ -1,25 +0,0 @@
/**
* abort-error.js
*
* AbortError interface for cancelled requests
*/
/**
* Create AbortError instance
*
* @param String message Error message for human
* @return AbortError
*/
export default function AbortError(message) {
Error.call(this, message);
this.type = 'aborted';
this.message = message;
// hide custom error implementation details from end-users
Error.captureStackTrace(this, this.constructor);
}
AbortError.prototype = Object.create(Error.prototype);
AbortError.prototype.constructor = AbortError;
AbortError.prototype.name = 'AbortError';

View File

@ -1,119 +0,0 @@
// Based on https://github.com/tmpvar/jsdom/blob/aa85b2abf07766ff7bf5c1f6daafb3726f2f2db5/lib/jsdom/living/blob.js
// (MIT licensed)
import Stream from 'stream';
// fix for "Readable" isn't a named export issue
const Readable = Stream.Readable;
export const BUFFER = Symbol('buffer');
const TYPE = Symbol('type');
export default class Blob {
constructor() {
this[TYPE] = '';
const blobParts = arguments[0];
const options = arguments[1];
const buffers = [];
let size = 0;
if (blobParts) {
const a = blobParts;
const length = Number(a.length);
for (let i = 0; i < length; i++) {
const element = a[i];
let buffer;
if (element instanceof Buffer) {
buffer = element;
} else if (ArrayBuffer.isView(element)) {
buffer = Buffer.from(element.buffer, element.byteOffset, element.byteLength);
} else if (element instanceof ArrayBuffer) {
buffer = Buffer.from(element);
} else if (element instanceof Blob) {
buffer = element[BUFFER];
} else {
buffer = Buffer.from(typeof element === 'string' ? element : String(element));
}
size += buffer.length;
buffers.push(buffer);
}
}
this[BUFFER] = Buffer.concat(buffers);
let type = options && options.type !== undefined && String(options.type).toLowerCase();
if (type && !/[^\u0020-\u007E]/.test(type)) {
this[TYPE] = type;
}
}
get size() {
return this[BUFFER].length;
}
get type() {
return this[TYPE];
}
text() {
return Promise.resolve(this[BUFFER].toString())
}
arrayBuffer() {
const buf = this[BUFFER];
const ab = buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength);
return Promise.resolve(ab);
}
stream() {
const readable = new Readable();
readable._read = () => {};
readable.push(this[BUFFER]);
readable.push(null);
return readable;
}
toString() {
return '[object Blob]'
}
slice() {
const size = this.size;
const start = arguments[0];
const end = arguments[1];
let relativeStart, relativeEnd;
if (start === undefined) {
relativeStart = 0;
} else if (start < 0) {
relativeStart = Math.max(size + start, 0);
} else {
relativeStart = Math.min(start, size);
}
if (end === undefined) {
relativeEnd = size;
} else if (end < 0) {
relativeEnd = Math.max(size + end, 0);
} else {
relativeEnd = Math.min(end, size);
}
const span = Math.max(relativeEnd - relativeStart, 0);
const buffer = this[BUFFER];
const slicedBuffer = buffer.slice(
relativeStart,
relativeStart + span
);
const blob = new Blob([], { type: arguments[2] });
blob[BUFFER] = slicedBuffer;
return blob;
}
}
Object.defineProperties(Blob.prototype, {
size: { enumerable: true },
type: { enumerable: true },
slice: { enumerable: true }
});
Object.defineProperty(Blob.prototype, Symbol.toStringTag, {
value: 'Blob',
writable: false,
enumerable: false,
configurable: true
});

View File

@ -1,23 +1,18 @@
/**
* body.js
* Body.js
*
* Body interface provides common methods for Request and Response
*/
import Stream from 'stream';
import Stream, {PassThrough} from 'stream';
import Blob, { BUFFER } from './blob.js';
import FetchError from './fetch-error.js';
let convert;
try { convert = require('encoding').convert; } catch(e) {}
import Blob from 'fetch-blob';
import FetchError from './errors/fetch-error';
import {isBlob, isURLSearchParams, isArrayBuffer, isAbortError} from './utils/is';
const INTERNALS = Symbol('Body internals');
// fix an issue where "PassThrough" isn't a named export for node <10
const PassThrough = Stream.PassThrough;
/**
* Body mixin
*
@ -31,29 +26,30 @@ export default function Body(body, {
size = 0,
timeout = 0
} = {}) {
if (body == null) {
// body is undefined or null
if (body === null) {
// Body is undefined or null
body = null;
} else if (isURLSearchParams(body)) {
// body is a URLSearchParams
// Body is a URLSearchParams
body = Buffer.from(body.toString());
} else if (isBlob(body)) {
// body is blob
// Body is blob
} else if (Buffer.isBuffer(body)) {
// body is Buffer
} else if (Object.prototype.toString.call(body) === '[object ArrayBuffer]') {
// body is ArrayBuffer
// Body is Buffer
} else if (isArrayBuffer(body)) {
// Body is ArrayBuffer
body = Buffer.from(body);
} else if (ArrayBuffer.isView(body)) {
// body is ArrayBufferView
// Body is ArrayBufferView
body = Buffer.from(body.buffer, body.byteOffset, body.byteLength);
} else if (body instanceof Stream) {
// body is stream
// Body is stream
} else {
// none of the above
// None of the above
// coerce to string then buffer
body = Buffer.from(String(body));
}
this[INTERNALS] = {
body,
disturbed: false,
@ -64,9 +60,9 @@ export default function Body(body, {
if (body instanceof Stream) {
body.on('error', err => {
const error = err.name === 'AbortError'
? err
: new FetchError(`Invalid response body while trying to fetch ${this.url}: ${err.message}`, 'system', err);
const error = isAbortError(err) ?
err :
new FetchError(`Invalid response body while trying to fetch ${this.url}: ${err.message}`, 'system', err);
this[INTERNALS].error = error;
});
}
@ -87,7 +83,7 @@ Body.prototype = {
* @return Promise
*/
arrayBuffer() {
return consumeBody.call(this).then(buf => buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength));
return consumeBody.call(this).then(({buffer, byteOffset, byteLength}) => buffer.slice(byteOffset, byteOffset + byteLength));
},
/**
@ -96,16 +92,11 @@ Body.prototype = {
* @return Promise
*/
blob() {
let ct = this.headers && this.headers.get('content-type') || '';
return consumeBody.call(this).then(buf => Object.assign(
// Prevent copying
new Blob([], {
type: ct.toLowerCase()
}),
{
[BUFFER]: buf
}
));
const ct = this.headers && this.headers.get('content-type') || this[INTERNALS].body && this[INTERNALS].body.type || '';
return consumeBody.call(this).then(buf => new Blob([], {
type: ct.toLowerCase(),
buffer: buf
}));
},
/**
@ -114,13 +105,7 @@ Body.prototype = {
* @return Promise
*/
json() {
return consumeBody.call(this).then((buffer) => {
try {
return JSON.parse(buffer.toString());
} catch (err) {
return Body.Promise.reject(new FetchError(`invalid json response body at ${this.url} reason: ${err.message}`, 'invalid-json'));
}
})
return consumeBody.call(this).then(buffer => JSON.parse(buffer.toString()));
},
/**
@ -139,33 +124,23 @@ Body.prototype = {
*/
buffer() {
return consumeBody.call(this);
},
/**
* Decode response as text, while automatically detecting the encoding and
* trying to decode to UTF-8 (non-spec api)
*
* @return Promise
*/
textConverted() {
return consumeBody.call(this).then(buffer => convertBody(buffer, this.headers));
}
};
// In browsers, all properties are enumerable.
Object.defineProperties(Body.prototype, {
body: { enumerable: true },
bodyUsed: { enumerable: true },
arrayBuffer: { enumerable: true },
blob: { enumerable: true },
json: { enumerable: true },
text: { enumerable: true }
body: {enumerable: true},
bodyUsed: {enumerable: true},
arrayBuffer: {enumerable: true},
blob: {enumerable: true},
json: {enumerable: true},
text: {enumerable: true}
});
Body.mixIn = function (proto) {
Body.mixIn = proto => {
for (const name of Object.getOwnPropertyNames(Body.prototype)) {
// istanbul ignore else: future proof
if (!(name in proto)) {
if (!Object.prototype.hasOwnProperty.call(proto, name)) {
const desc = Object.getOwnPropertyDescriptor(Body.prototype, name);
Object.defineProperty(proto, name, desc);
}
@ -190,19 +165,19 @@ function consumeBody() {
return Body.Promise.reject(this[INTERNALS].error);
}
let body = this.body;
let {body} = this;
// body is null
// Body is null
if (body === null) {
return Body.Promise.resolve(Buffer.alloc(0));
}
// body is blob
// Body is blob
if (isBlob(body)) {
body = body.stream();
}
// body is buffer
// Body is buffer
if (Buffer.isBuffer(body)) {
return Body.Promise.resolve(body);
}
@ -212,16 +187,16 @@ function consumeBody() {
return Body.Promise.resolve(Buffer.alloc(0));
}
// body is stream
// Body is stream
// get ready to actually consume the body
let accum = [];
const accum = [];
let accumBytes = 0;
let abort = false;
return new Body.Promise((resolve, reject) => {
let resTimeout;
// allow timeout on slow response body
// Allow timeout on slow response body
if (this.timeout) {
resTimeout = setTimeout(() => {
abort = true;
@ -229,14 +204,14 @@ function consumeBody() {
}, this.timeout);
}
// handle stream errors
// Handle stream errors
body.on('error', err => {
if (err.name === 'AbortError') {
// if the request was aborted, reject with this Error
if (isAbortError(err)) {
// If the request was aborted, reject with this Error
abort = true;
reject(err);
} else {
// other errors, such as incorrect content-encoding
// Other errors, such as incorrect content-encoding
reject(new FetchError(`Invalid response body while trying to fetch ${this.url}: ${err.message}`, 'system', err));
}
});
@ -265,148 +240,40 @@ function consumeBody() {
try {
resolve(Buffer.concat(accum, accumBytes));
} catch (err) {
// handle streams that have accumulated too much data (issue #414)
reject(new FetchError(`Could not create Buffer from response body for ${this.url}: ${err.message}`, 'system', err));
} catch (error) {
// Handle streams that have accumulated too much data (issue #414)
reject(new FetchError(`Could not create Buffer from response body for ${this.url}: ${error.message}`, 'system', error));
}
});
});
}
/**
* Detect buffer encoding and convert to target encoding
* ref: http://www.w3.org/TR/2011/WD-html5-20110113/parsing.html#determining-the-character-encoding
*
* @param Buffer buffer Incoming buffer
* @param String encoding Target encoding
* @return String
*/
function convertBody(buffer, headers) {
if (typeof convert !== 'function') {
throw new Error('The package `encoding` must be installed to use the textConverted() function');
}
const ct = headers.get('content-type');
let charset = 'utf-8';
let res, str;
// header
if (ct) {
res = /charset=([^;]*)/i.exec(ct);
}
// no charset in content type, peek at response body for at most 1024 bytes
str = buffer.slice(0, 1024).toString();
// html5
if (!res && str) {
res = /<meta.+?charset=(['"])(.+?)\1/i.exec(str);
}
// html4
if (!res && str) {
res = /<meta[\s]+?http-equiv=(['"])content-type\1[\s]+?content=(['"])(.+?)\2/i.exec(str);
if (!res) {
res = /<meta[\s]+?content=(['"])(.+?)\1[\s]+?http-equiv=(['"])content-type\3/i.exec(str);
if (res) {
res.pop(); // drop last quote
}
}
if (res) {
res = /charset=(.*)/i.exec(res.pop());
}
}
// xml
if (!res && str) {
res = /<\?xml.+?encoding=(['"])(.+?)\1/i.exec(str);
}
// found charset
if (res) {
charset = res.pop();
// prevent decode issues when sites use incorrect encoding
// ref: https://hsivonen.fi/encoding-menu/
if (charset === 'gb2312' || charset === 'gbk') {
charset = 'gb18030';
}
}
// turn raw buffers into a single utf-8 buffer
return convert(
buffer,
'UTF-8',
charset
).toString();
}
/**
* Detect a URLSearchParams object
* ref: https://github.com/bitinn/node-fetch/issues/296#issuecomment-307598143
*
* @param Object obj Object to detect by type or brand
* @return String
*/
function isURLSearchParams(obj) {
// Duck-typing as a necessary condition.
if (typeof obj !== 'object' ||
typeof obj.append !== 'function' ||
typeof obj.delete !== 'function' ||
typeof obj.get !== 'function' ||
typeof obj.getAll !== 'function' ||
typeof obj.has !== 'function' ||
typeof obj.set !== 'function') {
return false;
}
// Brand-checking and more duck-typing as optional condition.
return obj.constructor.name === 'URLSearchParams' ||
Object.prototype.toString.call(obj) === '[object URLSearchParams]' ||
typeof obj.sort === 'function';
}
/**
* Check if `obj` is a W3C `Blob` object (which `File` inherits from)
* @param {*} obj
* @return {boolean}
*/
function isBlob(obj) {
return typeof obj === 'object' &&
typeof obj.arrayBuffer === 'function' &&
typeof obj.type === 'string' &&
typeof obj.stream === 'function' &&
typeof obj.constructor === 'function' &&
typeof obj.constructor.name === 'string' &&
/^(Blob|File)$/.test(obj.constructor.name) &&
/^(Blob|File)$/.test(obj[Symbol.toStringTag])
}
/**
* Clone body given Res/Req instance
*
* @param Mixed instance Response or Request instance
* @param Mixed instance Response or Request instance
* @param String highWaterMark highWaterMark for both PassThrough body streams
* @return Mixed
*/
export function clone(instance) {
let p1, p2;
let body = instance.body;
export function clone(instance, highWaterMark) {
let p1;
let p2;
let {body} = instance;
// don't allow cloning a used body
// Don't allow cloning a used body
if (instance.bodyUsed) {
throw new Error('cannot clone body after it is used');
}
// check that body is a stream and not form-data object
// Check that body is a stream and not form-data object
// note: we can't clone the form-data object without having it as a dependency
if ((body instanceof Stream) && (typeof body.getBoundary !== 'function')) {
// tee instance body
p1 = new PassThrough();
p2 = new PassThrough();
// Tee instance body
p1 = new PassThrough({highWaterMark});
p2 = new PassThrough({highWaterMark});
body.pipe(p1);
body.pipe(p2);
// set instance body to teed body and return the other teed body
// Set instance body to teed body and return the other teed body
instance[INTERNALS].body = p1;
body = p2;
}
@ -421,41 +288,47 @@ export function clone(instance) {
*
* This function assumes that instance.body is present.
*
* @param Mixed instance Any options.body input
* @param {any} body Any options.body input
* @returns {string | null}
*/
export function extractContentType(body) {
// Body is null or undefined
if (body === null) {
// body is null
return null;
} else if (typeof body === 'string') {
// body is string
return 'text/plain;charset=UTF-8';
} else if (isURLSearchParams(body)) {
// body is a URLSearchParams
return 'application/x-www-form-urlencoded;charset=UTF-8';
} else if (isBlob(body)) {
// body is blob
return body.type || null;
} else if (Buffer.isBuffer(body)) {
// body is buffer
return null;
} else if (Object.prototype.toString.call(body) === '[object ArrayBuffer]') {
// body is ArrayBuffer
return null;
} else if (ArrayBuffer.isView(body)) {
// body is ArrayBufferView
return null;
} else if (typeof body.getBoundary === 'function') {
// detect form data input from form-data module
return `multipart/form-data;boundary=${body.getBoundary()}`;
} else if (body instanceof Stream) {
// body is stream
// can't really do much about this
return null;
} else {
// Body constructor defaults other things to string
}
// Body is string
if (typeof body === 'string') {
return 'text/plain;charset=UTF-8';
}
// Body is a URLSearchParams
if (isURLSearchParams(body)) {
return 'application/x-www-form-urlencoded;charset=UTF-8';
}
// Body is blob
if (isBlob(body)) {
return body.type || null;
}
// Body is a Buffer (Buffer, ArrayBuffer or ArrayBufferView)
if (Buffer.isBuffer(body) || isArrayBuffer(body) || ArrayBuffer.isView(body)) {
return null;
}
// Detect form data input from form-data module
if (body && typeof body.getBoundary === 'function') {
return `multipart/form-data;boundary=${body.getBoundary()}`;
}
// Body is stream - can't really do much about this
if (body instanceof Stream) {
return null;
}
// Body constructor defaults other things to string
return 'text/plain;charset=UTF-8';
}
/**
@ -464,56 +337,57 @@ export function extractContentType(body) {
*
* ref: https://fetch.spec.whatwg.org/#concept-body-total-bytes
*
* @param Body instance Instance of Body
* @return Number? Number of bytes, or null if not possible
* @param {any} obj.body Body object from the Body instance.
* @returns {number | null}
*/
export function getTotalBytes(instance) {
const {body} = instance;
export function getTotalBytes({body}) {
// Body is null or undefined
if (body === null) {
// body is null
return 0;
} else if (isBlob(body)) {
return body.size;
} else if (Buffer.isBuffer(body)) {
// body is buffer
return body.length;
} else if (body && typeof body.getLengthSync === 'function') {
// detect form data input from form-data module
if (body._lengthRetrievers && body._lengthRetrievers.length == 0 || // 1.x
body.hasKnownLength && body.hasKnownLength()) { // 2.x
return body.getLengthSync();
}
return null;
} else {
// body is stream
return null;
}
// Body is Blob
if (isBlob(body)) {
return body.size;
}
// Body is Buffer
if (Buffer.isBuffer(body)) {
return body.length;
}
// Detect form data input from form-data module
if (body && typeof body.getLengthSync === 'function') {
return body.hasKnownLength && body.hasKnownLength() ? body.getLengthSync() : null;
}
// Body is stream
return null;
}
/**
* Write a Body to a Node.js WritableStream (e.g. http.Request) object.
*
* @param Body instance Instance of Body
* @return Void
* @param {Stream.Writable} dest The stream to write to.
* @param obj.body Body object from the Body instance.
* @returns {void}
*/
export function writeToStream(dest, instance) {
const {body} = instance;
export function writeToStream(dest, {body}) {
if (body === null) {
// body is null
// Body is null
dest.end();
} else if (isBlob(body)) {
// Body is Blob
body.stream().pipe(dest);
} else if (Buffer.isBuffer(body)) {
// body is buffer
// Body is buffer
dest.write(body);
dest.end()
dest.end();
} else {
// body is stream
// Body is stream
body.pipe(dest);
}
}
// expose Promise
// Expose Promise
Body.Promise = global.Promise;

27
src/errors/abort-error.js Normal file
View File

@ -0,0 +1,27 @@
/**
* Abort-error.js
*
* AbortError interface for cancelled requests
*/
/**
* Create AbortError instance
*
* @param String message Error message for human
* @param String type Error type for machine
* @param String systemError For Node.js system error
* @return AbortError
*/
export default class AbortError extends Error {
constructor(message) {
super(message);
this.type = 'aborted';
this.message = message;
this.name = 'AbortError';
this[Symbol.toStringTag] = 'AbortError';
// Hide custom error implementation details from end-users
Error.captureStackTrace(this, this.constructor);
}
}

34
src/errors/fetch-error.js Normal file
View File

@ -0,0 +1,34 @@
/**
* Fetch-error.js
*
* FetchError interface for operational errors
*/
/**
* Create FetchError instance
*
* @param String message Error message for human
* @param String type Error type for machine
* @param Object systemError For Node.js system error
* @return FetchError
*/
export default class FetchError extends Error {
constructor(message, type, systemError) {
super(message);
this.message = message;
this.type = type;
this.name = 'FetchError';
this[Symbol.toStringTag] = 'FetchError';
// When err.type is `system`, err.erroredSysCall contains system error and err.code contains system error code
if (systemError) {
// eslint-disable-next-line no-multi-assign
this.code = this.errno = systemError.code;
this.erroredSysCall = systemError;
}
// Hide custom error implementation details from end-users
Error.captureStackTrace(this, this.constructor);
}
}

View File

@ -1,33 +0,0 @@
/**
* fetch-error.js
*
* FetchError interface for operational errors
*/
/**
* Create FetchError instance
*
* @param String message Error message for human
* @param String type Error type for machine
* @param String systemError For Node.js system error
* @return FetchError
*/
export default function FetchError(message, type, systemError) {
Error.call(this, message);
this.message = message;
this.type = type;
// when err.type is `system`, err.code contains system error code
if (systemError) {
this.code = this.errno = systemError.code;
}
// hide custom error implementation details from end-users
Error.captureStackTrace(this, this.constructor);
}
FetchError.prototype = Object.create(Error.prototype);
FetchError.prototype.constructor = FetchError;
FetchError.prototype.name = 'FetchError';

View File

@ -1,12 +1,12 @@
/**
* headers.js
* Headers.js
*
* Headers class offers convenient helpers
*/
const invalidTokenRegex = /[^\^_`a-zA-Z\-0-9!#$%&'*+.|~]/;
const invalidHeaderCharRegex = /[^\t\x20-\x7e\x80-\xff]/;
const invalidTokenRegex = /[^`\-\w!#$%&'*+.|~]/;
const invalidHeaderCharRegex = /[^\t\u0020-\u007E\u0080-\u00FF]/;
function validateName(name) {
name = `${name}`;
@ -37,6 +37,7 @@ function find(map, name) {
return key;
}
}
return undefined;
}
@ -66,33 +67,37 @@ export default class Headers {
// We don't worry about converting prop to ByteString here as append()
// will handle it.
// eslint-disable-next-line no-eq-null, eqeqeq
if (init == null) {
// no op
// No op
} else if (typeof init === 'object') {
const method = init[Symbol.iterator];
// eslint-disable-next-line no-eq-null, eqeqeq
if (method != null) {
if (typeof method !== 'function') {
throw new TypeError('Header pairs must be iterable');
}
// sequence<sequence<ByteString>>
// Sequence<sequence<ByteString>>
// Note: per spec we have to first exhaust the lists then process them
const pairs = [];
for (const pair of init) {
if (typeof pair !== 'object' || typeof pair[Symbol.iterator] !== 'function') {
throw new TypeError('Each header pair must be iterable');
}
pairs.push(Array.from(pair));
pairs.push([...pair]);
}
for (const pair of pairs) {
if (pair.length !== 2) {
throw new TypeError('Each header pair must be a name/value tuple');
}
this.append(pair[0], pair[1]);
}
} else {
// record<ByteString, ByteString>
// Record<ByteString, ByteString>
for (const key of Object.keys(init)) {
const value = init[key];
this.append(key, value);
@ -117,7 +122,12 @@ export default class Headers {
return null;
}
return this[MAP][key].join(', ');
let value = this[MAP][key].join(', ');
if (name.toLowerCase() === 'content-encoding') {
value = value.toLowerCase();
}
return value;
}
/**
@ -199,7 +209,7 @@ export default class Headers {
if (key !== undefined) {
delete this[MAP][key];
}
};
}
/**
* Return raw headers (non-spec api)
@ -249,15 +259,15 @@ Object.defineProperty(Headers.prototype, Symbol.toStringTag, {
});
Object.defineProperties(Headers.prototype, {
get: { enumerable: true },
forEach: { enumerable: true },
set: { enumerable: true },
append: { enumerable: true },
has: { enumerable: true },
delete: { enumerable: true },
keys: { enumerable: true },
values: { enumerable: true },
entries: { enumerable: true }
get: {enumerable: true},
forEach: {enumerable: true},
set: {enumerable: true},
append: {enumerable: true},
has: {enumerable: true},
delete: {enumerable: true},
keys: {enumerable: true},
values: {enumerable: true},
entries: {enumerable: true}
});
function getHeaders(headers, kind = 'key+value') {
@ -265,9 +275,9 @@ function getHeaders(headers, kind = 'key+value') {
return keys.map(
kind === 'key' ?
k => k.toLowerCase() :
kind === 'value' ?
(kind === 'value' ?
k => headers[MAP][k].join(', ') :
k => [k.toLowerCase(), headers[MAP][k].join(', ')]
k => [k.toLowerCase(), headers[MAP][k].join(', ')])
);
}
@ -297,8 +307,8 @@ const HeadersIteratorPrototype = Object.setPrototypeOf({
index
} = this[INTERNAL];
const values = getHeaders(target, kind);
const len = values.length;
if (index >= len) {
const length_ = values.length;
if (index >= length_) {
return {
value: undefined,
done: true
@ -330,16 +340,16 @@ Object.defineProperty(HeadersIteratorPrototype, Symbol.toStringTag, {
* @return Object
*/
export function exportNodeCompatibleHeaders(headers) {
const obj = Object.assign({ __proto__: null }, headers[MAP]);
const object = {__proto__: null, ...headers[MAP]};
// http.request() only supports string as Host header. This hack makes
// Http.request() only supports string as Host header. This hack makes
// specifying custom Host header possible.
const hostHeaderKey = find(headers[MAP], 'Host');
if (hostHeaderKey !== undefined) {
obj[hostHeaderKey] = obj[hostHeaderKey][0];
object[hostHeaderKey] = object[hostHeaderKey][0];
}
return obj;
return object;
}
/**
@ -349,26 +359,29 @@ export function exportNodeCompatibleHeaders(headers) {
* @param Object obj Object of headers
* @return Headers
*/
export function createHeadersLenient(obj) {
export function createHeadersLenient(object) {
const headers = new Headers();
for (const name of Object.keys(obj)) {
for (const name of Object.keys(object)) {
if (invalidTokenRegex.test(name)) {
continue;
}
if (Array.isArray(obj[name])) {
for (const val of obj[name]) {
if (invalidHeaderCharRegex.test(val)) {
if (Array.isArray(object[name])) {
for (const value of object[name]) {
if (invalidHeaderCharRegex.test(value)) {
continue;
}
if (headers[MAP][name] === undefined) {
headers[MAP][name] = [val];
headers[MAP][name] = [value];
} else {
headers[MAP][name].push(val);
headers[MAP][name].push(value);
}
}
} else if (!invalidHeaderCharRegex.test(obj[name])) {
headers[MAP][name] = [obj[name]];
} else if (!invalidHeaderCharRegex.test(object[name])) {
headers[MAP][name] = [object[name]];
}
}
return headers;
}

View File

@ -1,28 +1,23 @@
/**
* index.js
* Index.js
*
* a request API compatible with window.fetch
*
* All spec algorithm step numbers are based on https://fetch.spec.whatwg.org/commit-snapshots/ae716822cb3a61843226cd090eefc6589446c1d2/.
*/
import Url from 'url';
import http from 'http';
import https from 'https';
import zlib from 'zlib';
import Stream from 'stream';
import Stream, {PassThrough, pipeline as pump} from 'stream';
import dataURIToBuffer from 'data-uri-to-buffer';
import Body, { writeToStream, getTotalBytes } from './body';
import Body, {writeToStream, getTotalBytes} from './body';
import Response from './response';
import Headers, { createHeadersLenient } from './headers';
import Request, { getNodeRequestOptions } from './request';
import FetchError from './fetch-error';
import AbortError from './abort-error';
// fix an issue where "PassThrough", "resolve" aren't a named export for node <10
const PassThrough = Stream.PassThrough;
const resolve_url = Url.resolve;
import Headers, {createHeadersLenient} from './headers';
import Request, {getNodeRequestOptions} from './request';
import FetchError from './errors/fetch-error';
import AbortError from './errors/abort-error';
/**
* Fetch function
@ -31,45 +26,53 @@ const resolve_url = Url.resolve;
* @param Object opts Fetch options
* @return Promise
*/
export default function fetch(url, opts) {
// allow custom promise
export default function fetch(url, options_) {
// Allow custom promise
if (!fetch.Promise) {
throw new Error('native promise missing, set fetch.Promise to your favorite alternative');
}
if (/^data:/.test(url)) {
const request = new Request(url, opts);
try {
const data = Buffer.from(url.split(',')[1], 'base64')
const res = new Response(data.body, { headers: { 'Content-Type': data.mimeType || url.match(/^data:(.+);base64,.*$/)[1] } });
return fetch.Promise.resolve(res);
} catch (err) {
return fetch.Promise.reject(new FetchError(`[${request.method}] ${request.url} invalid URL, ${err.message}`, 'system', err));
}
// Regex for data uri
const dataUriRegex = /^\s*data:([a-z]+\/[a-z]+(;[a-z-]+=[a-z-]+)?)?(;base64)?,[\w!$&',()*+;=\-.~:@/?%\s]*\s*$/i;
// If valid data uri
if (dataUriRegex.test(url)) {
const data = dataURIToBuffer(url);
const res = new Response(data, {headers: {'Content-Type': data.type}});
return fetch.Promise.resolve(res);
}
// If invalid data uri
if (url.toString().startsWith('data:')) {
const request = new Request(url, options_);
return fetch.Promise.reject(new FetchError(`[${request.method}] ${request.url} invalid URL`, 'system'));
}
Body.Promise = fetch.Promise;
// wrap http.request into fetch
// Wrap http.request into fetch
return new fetch.Promise((resolve, reject) => {
// build request object
const request = new Request(url, opts);
// Build request object
const request = new Request(url, options_);
const options = getNodeRequestOptions(request);
const send = (options.protocol === 'https:' ? https : http).request;
const { signal } = request;
const {signal} = request;
let response = null;
const abort = () => {
let error = new AbortError('The user aborted a request.');
const abort = () => {
const error = new AbortError('The operation was aborted.');
reject(error);
if (request.body && request.body instanceof Stream.Readable) {
request.body.destroy(error);
}
if (!response || !response.body) return;
if (!response || !response.body) {
return;
}
response.body.emit('error', error);
}
};
if (signal && signal.aborted) {
abort();
@ -79,39 +82,35 @@ export default function fetch(url, opts) {
const abortAndFinalize = () => {
abort();
finalize();
}
};
// send request
const req = send(options);
let reqTimeout;
// Send request
const request_ = send(options);
if (signal) {
signal.addEventListener('abort', abortAndFinalize);
}
function finalize() {
req.abort();
if (signal) signal.removeEventListener('abort', abortAndFinalize);
clearTimeout(reqTimeout);
request_.abort();
if (signal) {
signal.removeEventListener('abort', abortAndFinalize);
}
}
if (request.timeout) {
req.once('socket', socket => {
reqTimeout = setTimeout(() => {
reject(new FetchError(`network timeout at: ${request.url}`, 'request-timeout'));
finalize();
}, request.timeout);
request_.setTimeout(request.timeout, () => {
finalize();
reject(new FetchError(`network timeout at: ${request.url}`, 'request-timeout'));
});
}
req.on('error', err => {
request_.on('error', err => {
reject(new FetchError(`request to ${request.url} failed, reason: ${err.message}`, 'system', err));
finalize();
});
req.on('response', res => {
clearTimeout(reqTimeout);
request_.on('response', res => {
const headers = createHeadersLenient(res.headers);
// HTTP fetch step 5
@ -120,7 +119,7 @@ export default function fetch(url, opts) {
const location = headers.get('Location');
// HTTP fetch step 5.3
const locationURL = location === null ? null : resolve_url(request.url, location);
const locationURL = location === null ? null : new URL(location, request.url);
// HTTP fetch step 5.5
switch (request.redirect) {
@ -129,18 +128,19 @@ export default function fetch(url, opts) {
finalize();
return;
case 'manual':
// node-fetch-specific step: make manual redirect a bit easier to use by setting the Location header value to the resolved URL.
// Node-fetch-specific step: make manual redirect a bit easier to use by setting the Location header value to the resolved URL.
if (locationURL !== null) {
// handle corrupted header
// Handle corrupted header
try {
headers.set('Location', locationURL);
} catch (err) {
} catch (error) {
// istanbul ignore next: nodejs server prevent invalid response headers, we can't test this through normal request
reject(err);
reject(error);
}
}
break;
case 'follow':
case 'follow': {
// HTTP-redirect fetch step 2
if (locationURL === null) {
break;
@ -155,7 +155,7 @@ export default function fetch(url, opts) {
// HTTP-redirect fetch step 6 (counter increment)
// Create a new Request object.
const requestOpts = {
const requestOptions = {
headers: new Headers(request.headers),
follow: request.follow,
counter: request.counter + 1,
@ -176,32 +176,42 @@ export default function fetch(url, opts) {
// HTTP-redirect fetch step 11
if (res.statusCode === 303 || ((res.statusCode === 301 || res.statusCode === 302) && request.method === 'POST')) {
requestOpts.method = 'GET';
requestOpts.body = undefined;
requestOpts.headers.delete('content-length');
requestOptions.method = 'GET';
requestOptions.body = undefined;
requestOptions.headers.delete('content-length');
}
// HTTP-redirect fetch step 15
resolve(fetch(new Request(locationURL, requestOpts)));
resolve(fetch(new Request(locationURL, requestOptions)));
finalize();
return;
}
default:
// Do nothing
}
}
// prepare response
// Prepare response
res.once('end', () => {
if (signal) signal.removeEventListener('abort', abortAndFinalize);
if (signal) {
signal.removeEventListener('abort', abortAndFinalize);
}
});
let body = res.pipe(new PassThrough());
const response_options = {
let body = pump(res, new PassThrough(), error => {
reject(error);
});
const responseOptions = {
url: request.url,
status: res.statusCode,
statusText: res.statusMessage,
headers: headers,
headers,
size: request.size,
timeout: request.timeout,
counter: request.counter
counter: request.counter,
highWaterMark: request.highWaterMark
};
// HTTP-network fetch step 12.1.1.3
@ -216,7 +226,7 @@ export default function fetch(url, opts) {
// 4. no content response (204)
// 5. content not modified response (304)
if (!request.compress || request.method === 'HEAD' || codings === null || res.statusCode === 204 || res.statusCode === 304) {
response = new Response(body, response_options);
response = new Response(body, responseOptions);
resolve(response);
return;
}
@ -231,49 +241,59 @@ export default function fetch(url, opts) {
finishFlush: zlib.Z_SYNC_FLUSH
};
// for gzip
if (codings == 'gzip' || codings == 'x-gzip') {
body = body.pipe(zlib.createGunzip(zlibOptions));
response = new Response(body, response_options);
// For gzip
if (codings === 'gzip' || codings === 'x-gzip') {
body = pump(body, zlib.createGunzip(zlibOptions), error => {
reject(error);
});
response = new Response(body, responseOptions);
resolve(response);
return;
}
// for deflate
if (codings == 'deflate' || codings == 'x-deflate') {
// handle the infamous raw deflate response from old servers
// For deflate
if (codings === 'deflate' || codings === 'x-deflate') {
// Handle the infamous raw deflate response from old servers
// a hack for old IIS and Apache servers
const raw = res.pipe(new PassThrough());
const raw = pump(res, new PassThrough(), error => {
reject(error);
});
raw.once('data', chunk => {
// see http://stackoverflow.com/questions/37519828
// See http://stackoverflow.com/questions/37519828
if ((chunk[0] & 0x0F) === 0x08) {
body = body.pipe(zlib.createInflate());
body = pump(body, zlib.createInflate(), error => {
reject(error);
});
} else {
body = body.pipe(zlib.createInflateRaw());
body = pump(body, zlib.createInflateRaw(), error => {
reject(error);
});
}
response = new Response(body, response_options);
response = new Response(body, responseOptions);
resolve(response);
});
return;
}
// for br
if (codings == 'br' && typeof zlib.createBrotliDecompress === 'function') {
body = body.pipe(zlib.createBrotliDecompress());
response = new Response(body, response_options);
// For br
if (codings === 'br' && typeof zlib.createBrotliDecompress === 'function') {
body = pump(body, zlib.createBrotliDecompress(), error => {
reject(error);
});
response = new Response(body, responseOptions);
resolve(response);
return;
}
// otherwise, use response as-is
response = new Response(body, response_options);
// Otherwise, use response as-is
response = new Response(body, responseOptions);
resolve(response);
});
writeToStream(req, request);
writeToStream(request_, request);
});
};
}
/**
* Redirect code matching
@ -281,9 +301,9 @@ export default function fetch(url, opts) {
* @param Number code Status code
* @return Boolean
*/
fetch.isRedirect = code => code === 301 || code === 302 || code === 303 || code === 307 || code === 308;
fetch.isRedirect = code => [301, 302, 303, 307, 308].includes(code);
// expose Promise
// Expose Promise
fetch.Promise = global.Promise;
export {
Headers,

View File

@ -1,45 +1,53 @@
/**
* request.js
* Request.js
*
* Request class contains server only options
*
* All spec algorithm step numbers are based on https://fetch.spec.whatwg.org/commit-snapshots/ae716822cb3a61843226cd090eefc6589446c1d2/.
*/
import Url from 'url';
import {format as formatUrl} from 'url';
import Stream from 'stream';
import Headers, { exportNodeCompatibleHeaders } from './headers.js';
import Body, { clone, extractContentType, getTotalBytes } from './body';
import Headers, {exportNodeCompatibleHeaders} from './headers';
import Body, {clone, extractContentType, getTotalBytes} from './body';
import {isAbortSignal} from './utils/is';
const INTERNALS = Symbol('Request internals');
// fix an issue where "format", "parse" aren't a named export for node <10
const parse_url = Url.parse;
const format_url = Url.format;
const streamDestructionSupported = 'destroy' in Stream.Readable.prototype;
/**
* Check if a value is an instance of Request.
* Check if `obj` is an instance of Request.
*
* @param Mixed input
* @return Boolean
* @param {*} obj
* @return {boolean}
*/
function isRequest(input) {
function isRequest(object) {
return (
typeof input === 'object' &&
typeof input[INTERNALS] === 'object'
typeof object === 'object' &&
typeof object[INTERNALS] === 'object'
);
}
function isAbortSignal(signal) {
const proto = (
signal
&& typeof signal === 'object'
&& Object.getPrototypeOf(signal)
);
return !!(proto && proto.constructor.name === 'AbortSignal');
/**
* Wrapper around `new URL` to handle relative URLs (https://github.com/nodejs/node/issues/12682)
*
* @param {string} urlStr
* @return {void}
*/
function parseURL(urlString) {
/*
Check whether the URL is absolute or not
Scheme: https://tools.ietf.org/html/rfc3986#section-3.1
Absolute URL: https://tools.ietf.org/html/rfc3986#section-4.3
*/
if (/^[a-zA-Z][a-zA-Z\d+\-.]*:/.exec(urlString)) {
return new URL(urlString);
}
throw new TypeError('Only absolute URLs are supported');
}
/**
@ -53,35 +61,38 @@ export default class Request {
constructor(input, init = {}) {
let parsedURL;
// normalize input
// Normalize input and force URL to be encoded as UTF-8 (https://github.com/bitinn/node-fetch/issues/245)
if (!isRequest(input)) {
if (input && input.href) {
// in order to support Node.js' Url objects; though WHATWG's URL objects
// In order to support Node.js' Url objects; though WHATWG's URL objects
// will fall into this branch also (since their `toString()` will return
// `href` property anyway)
parsedURL = parse_url(input.href);
parsedURL = parseURL(input.href);
} else {
// coerce input to a string before attempting to parse
parsedURL = parse_url(`${input}`);
// Coerce input to a string before attempting to parse
parsedURL = parseURL(`${input}`);
}
input = {};
} else {
parsedURL = parse_url(input.url);
parsedURL = parseURL(input.url);
}
let method = init.method || input.method || 'GET';
method = method.toUpperCase();
// eslint-disable-next-line no-eq-null, eqeqeq
if ((init.body != null || isRequest(input) && input.body !== null) &&
(method === 'GET' || method === 'HEAD')) {
throw new TypeError('Request with GET/HEAD method cannot have body');
}
let inputBody = init.body != null ?
// eslint-disable-next-line no-eq-null, eqeqeq
const inputBody = init.body != null ?
init.body :
isRequest(input) && input.body !== null ?
(isRequest(input) && input.body !== null ?
clone(input) :
null;
null);
Body.call(this, inputBody, {
timeout: init.timeout || input.timeout || 0,
@ -90,19 +101,21 @@ export default class Request {
const headers = new Headers(init.headers || input.headers || {});
if (inputBody != null && !headers.has('Content-Type')) {
if (inputBody !== null && !headers.has('Content-Type')) {
const contentType = extractContentType(inputBody);
if (contentType) {
headers.append('Content-Type', contentType);
}
}
let signal = isRequest(input)
? input.signal
: null;
if ('signal' in init) signal = init.signal
let signal = isRequest(input) ?
input.signal :
null;
if ('signal' in init) {
signal = init.signal;
}
if (signal != null && !isAbortSignal(signal)) {
if (signal !== null && !isAbortSignal(signal)) {
throw new TypeError('Expected signal to be an instanceof AbortSignal');
}
@ -111,18 +124,19 @@ export default class Request {
redirect: init.redirect || input.redirect || 'follow',
headers,
parsedURL,
signal,
signal
};
// node-fetch-only options
// Node-fetch-only options
this.follow = init.follow !== undefined ?
init.follow : input.follow !== undefined ?
input.follow : 20;
init.follow : (input.follow !== undefined ?
input.follow : 20);
this.compress = init.compress !== undefined ?
init.compress : input.compress !== undefined ?
input.compress : true;
init.compress : (input.compress !== undefined ?
input.compress : true);
this.counter = init.counter || input.counter || 0;
this.agent = init.agent || input.agent;
this.highWaterMark = init.highWaterMark || input.highWaterMark;
}
get method() {
@ -130,7 +144,7 @@ export default class Request {
}
get url() {
return format_url(this[INTERNALS].parsedURL);
return formatUrl(this[INTERNALS].parsedURL);
}
get headers() {
@ -165,12 +179,12 @@ Object.defineProperty(Request.prototype, Symbol.toStringTag, {
});
Object.defineProperties(Request.prototype, {
method: { enumerable: true },
url: { enumerable: true },
headers: { enumerable: true },
redirect: { enumerable: true },
clone: { enumerable: true },
signal: { enumerable: true },
method: {enumerable: true},
url: {enumerable: true},
headers: {enumerable: true},
redirect: {enumerable: true},
clone: {enumerable: true},
signal: {enumerable: true}
});
/**
@ -180,10 +194,10 @@ Object.defineProperties(Request.prototype, {
* @return Object The options object to be passed to http.request
*/
export function getNodeRequestOptions(request) {
const parsedURL = request[INTERNALS].parsedURL;
const {parsedURL} = request[INTERNALS];
const headers = new Headers(request[INTERNALS].headers);
// fetch step 1.3
// Fetch step 1.3
if (!headers.has('Accept')) {
headers.set('Accept', '*/*');
}
@ -198,24 +212,26 @@ export function getNodeRequestOptions(request) {
}
if (
request.signal
&& request.body instanceof Stream.Readable
&& !streamDestructionSupported
request.signal &&
request.body instanceof Stream.Readable &&
!streamDestructionSupported
) {
throw new Error('Cancellation of streamed requests with AbortSignal is not supported in node < 8');
throw new Error('Cancellation of streamed requests with AbortSignal is not supported');
}
// HTTP-network-or-cache fetch steps 2.4-2.7
let contentLengthValue = null;
if (request.body == null && /^(POST|PUT)$/i.test(request.method)) {
if (request.body === null && /^(post|put)$/i.test(request.method)) {
contentLengthValue = '0';
}
if (request.body != null) {
if (request.body !== null) {
const totalBytes = getTotalBytes(request);
if (typeof totalBytes === 'number') {
contentLengthValue = String(totalBytes);
}
}
if (contentLengthValue) {
headers.set('Content-Length', contentLengthValue);
}
@ -230,7 +246,7 @@ export function getNodeRequestOptions(request) {
headers.set('Accept-Encoding', 'gzip,deflate');
}
let agent = request.agent;
let {agent} = request;
if (typeof agent === 'function') {
agent = agent(parsedURL);
}
@ -242,9 +258,21 @@ export function getNodeRequestOptions(request) {
// HTTP-network fetch step 4.2
// chunked encoding is handled by Node.js
return Object.assign({}, parsedURL, {
// manually spread the URL object instead of spread syntax
const requestOptions = {
path: parsedURL.pathname,
pathname: parsedURL.pathname,
hostname: parsedURL.hostname,
protocol: parsedURL.protocol,
port: parsedURL.port,
hash: parsedURL.hash,
search: parsedURL.search,
query: parsedURL.query,
href: parsedURL.href,
method: request.method,
headers: exportNodeCompatibleHeaders(headers),
agent
});
};
return requestOptions;
}

View File

@ -1,20 +1,14 @@
/**
* response.js
* Response.js
*
* Response class provides content decoding
*/
import http from 'http';
import Headers from './headers.js';
import Body, { clone, extractContentType } from './body';
import Headers from './headers';
import Body, {clone, extractContentType} from './body';
const INTERNALS = Symbol('Response internals');
// fix an issue where "STATUS_CODES" aren't a named export for node <10
const STATUS_CODES = http.STATUS_CODES;
/**
* Response class
*
@ -23,13 +17,13 @@ const STATUS_CODES = http.STATUS_CODES;
* @return Void
*/
export default class Response {
constructor(body = null, opts = {}) {
Body.call(this, body, opts);
constructor(body = null, options = {}) {
Body.call(this, body, options);
const status = opts.status || 200;
const headers = new Headers(opts.headers)
const status = options.status || 200;
const headers = new Headers(options.headers);
if (body != null && !headers.has('Content-Type')) {
if (body !== null && !headers.has('Content-Type')) {
const contentType = extractContentType(body);
if (contentType) {
headers.append('Content-Type', contentType);
@ -37,11 +31,12 @@ export default class Response {
}
this[INTERNALS] = {
url: opts.url,
url: options.url,
status,
statusText: opts.statusText || STATUS_CODES[status],
statusText: options.statusText || '',
headers,
counter: opts.counter
counter: options.counter,
highWaterMark: options.highWaterMark
};
}
@ -72,19 +67,43 @@ export default class Response {
return this[INTERNALS].headers;
}
get highWaterMark() {
return this[INTERNALS].highWaterMark;
}
/**
* Clone this response
*
* @return Response
*/
clone() {
return new Response(clone(this), {
return new Response(clone(this, this.highWaterMark), {
url: this.url,
status: this.status,
statusText: this.statusText,
headers: this.headers,
ok: this.ok,
redirected: this.redirected
redirected: this.redirected,
size: this.size,
timeout: this.timeout
});
}
/**
* @param {string} url The URL that the new response is to originate from.
* @param {number} status An optional status code for the response (e.g., 302.)
* @returns {Response} A Response object.
*/
static redirect(url, status = 302) {
if (![301, 302, 303, 307, 308].includes(status)) {
throw new RangeError('Failed to execute "redirect" on "response": Invalid status code');
}
return new Response(null, {
headers: {
location: new URL(url).toString()
},
status
});
}
}
@ -92,13 +111,13 @@ export default class Response {
Body.mixIn(Response.prototype);
Object.defineProperties(Response.prototype, {
url: { enumerable: true },
status: { enumerable: true },
ok: { enumerable: true },
redirected: { enumerable: true },
statusText: { enumerable: true },
headers: { enumerable: true },
clone: { enumerable: true }
url: {enumerable: true},
status: {enumerable: true},
ok: {enumerable: true},
redirected: {enumerable: true},
statusText: {enumerable: true},
headers: {enumerable: true},
clone: {enumerable: true}
});
Object.defineProperty(Response.prototype, Symbol.toStringTag, {

78
src/utils/is.js Normal file
View File

@ -0,0 +1,78 @@
/**
* Is.js
*
* Object type checks.
*/
const NAME = Symbol.toStringTag;
/**
* Check if `obj` is a URLSearchParams object
* ref: https://github.com/node-fetch/node-fetch/issues/296#issuecomment-307598143
*
* @param {*} obj
* @return {boolean}
*/
export function isURLSearchParams(object) {
return (
typeof object === 'object' &&
typeof object.append === 'function' &&
typeof object.delete === 'function' &&
typeof object.get === 'function' &&
typeof object.getAll === 'function' &&
typeof object.has === 'function' &&
typeof object.set === 'function' &&
typeof object.sort === 'function' &&
object[NAME] === 'URLSearchParams'
);
}
/**
* Check if `obj` is a W3C `Blob` object (which `File` inherits from)
*
* @param {*} obj
* @return {boolean}
*/
export function isBlob(object) {
return (
typeof object === 'object' &&
typeof object.arrayBuffer === 'function' &&
typeof object.type === 'string' &&
typeof object.stream === 'function' &&
typeof object.constructor === 'function' &&
/^(Blob|File)$/.test(object[NAME])
);
}
/**
* Check if `obj` is an instance of AbortSignal.
*
* @param {*} obj
* @return {boolean}
*/
export function isAbortSignal(object) {
return (
typeof object === 'object' &&
object[NAME] === 'AbortSignal'
);
}
/**
* Check if `obj` is an instance of ArrayBuffer.
*
* @param {*} obj
* @return {boolean}
*/
export function isArrayBuffer(object) {
return object[NAME] === 'ArrayBuffer';
}
/**
* Check if `obj` is an instance of AbortError.
*
* @param {*} obj
* @return {boolean}
*/
export function isAbortError(object) {
return object[NAME] === 'AbortError';
}

34
test/external-encoding.js Normal file
View File

@ -0,0 +1,34 @@
import fetch from '../src';
import chai from 'chai';
const {expect} = chai;
describe('external encoding', () => {
describe('data uri', () => {
it('should accept data uri', () => {
return fetch('data:image/gif;base64,R0lGODlhAQABAIAAAAUEBAAAACwAAAAAAQABAAACAkQBADs=').then(r => {
expect(r.status).to.equal(200);
expect(r.headers.get('Content-Type')).to.equal('image/gif');
return r.buffer().then(b => {
expect(b).to.be.an.instanceOf(Buffer);
});
});
});
it('should accept data uri of plain text', () => {
return fetch('data:,Hello%20World!').then(r => {
expect(r.status).to.equal(200);
expect(r.headers.get('Content-Type')).to.equal('text/plain');
return r.text().then(t => expect(t).to.equal('Hello World!'));
});
});
it('should reject invalid data uri', () => {
return fetch('data:@@@@').catch(error => {
expect(error).to.exist;
expect(error.message).to.include('invalid URL');
});
});
});
});

232
test/headers.js Normal file
View File

@ -0,0 +1,232 @@
import {Headers} from '../src';
import chai from 'chai';
const {expect} = chai;
describe('Headers', () => {
it('should have attributes conforming to Web IDL', () => {
const headers = new Headers();
expect(Object.getOwnPropertyNames(headers)).to.be.empty;
const enumerableProperties = [];
for (const property in headers) {
enumerableProperties.push(property);
}
for (const toCheck of [
'append',
'delete',
'entries',
'forEach',
'get',
'has',
'keys',
'set',
'values'
]) {
expect(enumerableProperties).to.contain(toCheck);
}
});
it('should allow iterating through all headers with forEach', () => {
const headers = new Headers([
['b', '2'],
['c', '4'],
['b', '3'],
['a', '1']
]);
expect(headers).to.have.property('forEach');
const result = [];
headers.forEach((value, key) => {
result.push([key, value]);
});
expect(result).to.deep.equal([
['a', '1'],
['b', '2, 3'],
['c', '4']
]);
});
it('should allow iterating through all headers with for-of loop', () => {
const headers = new Headers([
['b', '2'],
['c', '4'],
['a', '1']
]);
headers.append('b', '3');
expect(headers).to.be.iterable;
const result = [];
for (const pair of headers) {
result.push(pair);
}
expect(result).to.deep.equal([
['a', '1'],
['b', '2, 3'],
['c', '4']
]);
});
it('should allow iterating through all headers with entries()', () => {
const headers = new Headers([
['b', '2'],
['c', '4'],
['a', '1']
]);
headers.append('b', '3');
expect(headers.entries()).to.be.iterable
.and.to.deep.iterate.over([
['a', '1'],
['b', '2, 3'],
['c', '4']
]);
});
it('should allow iterating through all headers with keys()', () => {
const headers = new Headers([
['b', '2'],
['c', '4'],
['a', '1']
]);
headers.append('b', '3');
expect(headers.keys()).to.be.iterable
.and.to.iterate.over(['a', 'b', 'c']);
});
it('should allow iterating through all headers with values()', () => {
const headers = new Headers([
['b', '2'],
['c', '4'],
['a', '1']
]);
headers.append('b', '3');
expect(headers.values()).to.be.iterable
.and.to.iterate.over(['1', '2, 3', '4']);
});
it('should reject illegal header', () => {
const headers = new Headers();
expect(() => new Headers({'He y': 'ok'})).to.throw(TypeError);
expect(() => new Headers({'Hé-y': 'ok'})).to.throw(TypeError);
expect(() => new Headers({'He-y': 'ăk'})).to.throw(TypeError);
expect(() => headers.append('Hé-y', 'ok')).to.throw(TypeError);
expect(() => headers.delete('Hé-y')).to.throw(TypeError);
expect(() => headers.get('Hé-y')).to.throw(TypeError);
expect(() => headers.has('Hé-y')).to.throw(TypeError);
expect(() => headers.set('Hé-y', 'ok')).to.throw(TypeError);
// Should reject empty header
expect(() => headers.append('', 'ok')).to.throw(TypeError);
});
it('should ignore unsupported attributes while reading headers', () => {
const FakeHeader = function () { };
// Prototypes are currently ignored
// This might change in the future: #181
FakeHeader.prototype.z = 'fake';
const res = new FakeHeader();
res.a = 'string';
res.b = ['1', '2'];
res.c = '';
res.d = [];
res.e = 1;
res.f = [1, 2];
res.g = {a: 1};
res.h = undefined;
res.i = null;
res.j = NaN;
res.k = true;
res.l = false;
res.m = Buffer.from('test');
const h1 = new Headers(res);
h1.set('n', [1, 2]);
h1.append('n', ['3', 4]);
const h1Raw = h1.raw();
expect(h1Raw.a).to.include('string');
expect(h1Raw.b).to.include('1,2');
expect(h1Raw.c).to.include('');
expect(h1Raw.d).to.include('');
expect(h1Raw.e).to.include('1');
expect(h1Raw.f).to.include('1,2');
expect(h1Raw.g).to.include('[object Object]');
expect(h1Raw.h).to.include('undefined');
expect(h1Raw.i).to.include('null');
expect(h1Raw.j).to.include('NaN');
expect(h1Raw.k).to.include('true');
expect(h1Raw.l).to.include('false');
expect(h1Raw.m).to.include('test');
expect(h1Raw.n).to.include('1,2');
expect(h1Raw.n).to.include('3,4');
expect(h1Raw.z).to.be.undefined;
});
it('should wrap headers', () => {
const h1 = new Headers({
a: '1'
});
const h1Raw = h1.raw();
const h2 = new Headers(h1);
h2.set('b', '1');
const h2Raw = h2.raw();
const h3 = new Headers(h2);
h3.append('a', '2');
const h3Raw = h3.raw();
expect(h1Raw.a).to.include('1');
expect(h1Raw.a).to.not.include('2');
expect(h2Raw.a).to.include('1');
expect(h2Raw.a).to.not.include('2');
expect(h2Raw.b).to.include('1');
expect(h3Raw.a).to.include('1');
expect(h3Raw.a).to.include('2');
expect(h3Raw.b).to.include('1');
});
it('should accept headers as an iterable of tuples', () => {
let headers;
headers = new Headers([
['a', '1'],
['b', '2'],
['a', '3']
]);
expect(headers.get('a')).to.equal('1, 3');
expect(headers.get('b')).to.equal('2');
headers = new Headers([
new Set(['a', '1']),
['b', '2'],
new Map([['a', null], ['3', null]]).keys()
]);
expect(headers.get('a')).to.equal('1, 3');
expect(headers.get('b')).to.equal('2');
headers = new Headers(new Map([
['a', '1'],
['b', '2']
]));
expect(headers.get('a')).to.equal('1');
expect(headers.get('b')).to.equal('2');
});
it('should throw a TypeError if non-tuple exists in a headers initializer', () => {
expect(() => new Headers([['b', '2', 'huh?']])).to.throw(TypeError);
expect(() => new Headers(['b2'])).to.throw(TypeError);
expect(() => new Headers('b2')).to.throw(TypeError);
expect(() => new Headers({[Symbol.iterator]: 42})).to.throw(TypeError);
});
});

File diff suppressed because it is too large Load Diff

266
test/request.js Normal file
View File

@ -0,0 +1,266 @@
import * as stream from 'stream';
import * as http from 'http';
import {Request} from '../src';
import TestServer from './utils/server';
import {AbortController} from 'abortcontroller-polyfill/dist/abortcontroller';
import chai from 'chai';
import FormData from 'form-data';
import Blob from 'fetch-blob';
import resumer from 'resumer';
import stringToArrayBuffer from 'string-to-arraybuffer';
const {expect} = chai;
const local = new TestServer();
const base = `http://${local.hostname}:${local.port}/`;
describe('Request', () => {
it('should have attributes conforming to Web IDL', () => {
const request = new Request('https://github.com/');
const enumerableProperties = [];
for (const property in request) {
enumerableProperties.push(property);
}
for (const toCheck of [
'body',
'bodyUsed',
'arrayBuffer',
'blob',
'json',
'text',
'method',
'url',
'headers',
'redirect',
'clone',
'signal'
]) {
expect(enumerableProperties).to.contain(toCheck);
}
for (const toCheck of [
'body', 'bodyUsed', 'method', 'url', 'headers', 'redirect', 'signal'
]) {
expect(() => {
request[toCheck] = 'abc';
}).to.throw();
}
});
it('should support wrapping Request instance', () => {
const url = `${base}hello`;
const form = new FormData();
form.append('a', '1');
const {signal} = new AbortController();
const r1 = new Request(url, {
method: 'POST',
follow: 1,
body: form,
signal
});
const r2 = new Request(r1, {
follow: 2
});
expect(r2.url).to.equal(url);
expect(r2.method).to.equal('POST');
expect(r2.signal).to.equal(signal);
// Note that we didn't clone the body
expect(r2.body).to.equal(form);
expect(r1.follow).to.equal(1);
expect(r2.follow).to.equal(2);
expect(r1.counter).to.equal(0);
expect(r2.counter).to.equal(0);
});
it('should override signal on derived Request instances', () => {
const parentAbortController = new AbortController();
const derivedAbortController = new AbortController();
const parentRequest = new Request(`${base}hello`, {
signal: parentAbortController.signal
});
const derivedRequest = new Request(parentRequest, {
signal: derivedAbortController.signal
});
expect(parentRequest.signal).to.equal(parentAbortController.signal);
expect(derivedRequest.signal).to.equal(derivedAbortController.signal);
});
it('should allow removing signal on derived Request instances', () => {
const parentAbortController = new AbortController();
const parentRequest = new Request(`${base}hello`, {
signal: parentAbortController.signal
});
const derivedRequest = new Request(parentRequest, {
signal: null
});
expect(parentRequest.signal).to.equal(parentAbortController.signal);
expect(derivedRequest.signal).to.equal(null);
});
it('should throw error with GET/HEAD requests with body', () => {
expect(() => new Request('.', {body: ''}))
.to.throw(TypeError);
expect(() => new Request('.', {body: 'a'}))
.to.throw(TypeError);
expect(() => new Request('.', {body: '', method: 'HEAD'}))
.to.throw(TypeError);
expect(() => new Request('.', {body: 'a', method: 'HEAD'}))
.to.throw(TypeError);
expect(() => new Request('.', {body: 'a', method: 'get'}))
.to.throw(TypeError);
expect(() => new Request('.', {body: 'a', method: 'head'}))
.to.throw(TypeError);
});
it('should default to null as body', () => {
const request = new Request(base);
expect(request.body).to.equal(null);
return request.text().then(result => expect(result).to.equal(''));
});
it('should support parsing headers', () => {
const url = base;
const request = new Request(url, {
headers: {
a: '1'
}
});
expect(request.url).to.equal(url);
expect(request.headers.get('a')).to.equal('1');
});
it('should support arrayBuffer() method', () => {
const url = base;
const request = new Request(url, {
method: 'POST',
body: 'a=1'
});
expect(request.url).to.equal(url);
return request.arrayBuffer().then(result => {
expect(result).to.be.an.instanceOf(ArrayBuffer);
const string = String.fromCharCode.apply(null, new Uint8Array(result));
expect(string).to.equal('a=1');
});
});
it('should support text() method', () => {
const url = base;
const request = new Request(url, {
method: 'POST',
body: 'a=1'
});
expect(request.url).to.equal(url);
return request.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support json() method', () => {
const url = base;
const request = new Request(url, {
method: 'POST',
body: '{"a":1}'
});
expect(request.url).to.equal(url);
return request.json().then(result => {
expect(result.a).to.equal(1);
});
});
it('should support buffer() method', () => {
const url = base;
const request = new Request(url, {
method: 'POST',
body: 'a=1'
});
expect(request.url).to.equal(url);
return request.buffer().then(result => {
expect(result.toString()).to.equal('a=1');
});
});
it('should support blob() method', () => {
const url = base;
const request = new Request(url, {
method: 'POST',
body: Buffer.from('a=1')
});
expect(request.url).to.equal(url);
return request.blob().then(result => {
expect(result).to.be.an.instanceOf(Blob);
expect(result.size).to.equal(3);
expect(result.type).to.equal('');
});
});
it('should support clone() method', () => {
const url = base;
let body = resumer().queue('a=1').end();
body = body.pipe(new stream.PassThrough());
const agent = new http.Agent();
const {signal} = new AbortController();
const request = new Request(url, {
body,
method: 'POST',
redirect: 'manual',
headers: {
b: '2'
},
follow: 3,
compress: false,
agent,
signal
});
const cl = request.clone();
expect(cl.url).to.equal(url);
expect(cl.method).to.equal('POST');
expect(cl.redirect).to.equal('manual');
expect(cl.headers.get('b')).to.equal('2');
expect(cl.follow).to.equal(3);
expect(cl.compress).to.equal(false);
expect(cl.method).to.equal('POST');
expect(cl.counter).to.equal(0);
expect(cl.agent).to.equal(agent);
expect(cl.signal).to.equal(signal);
// Clone body shouldn't be the same body
expect(cl.body).to.not.equal(body);
return Promise.all([cl.text(), request.text()]).then(results => {
expect(results[0]).to.equal('a=1');
expect(results[1]).to.equal('a=1');
});
});
it('should support ArrayBuffer as body', () => {
const request = new Request(base, {
method: 'POST',
body: stringToArrayBuffer('a=1')
});
return request.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support Uint8Array as body', () => {
const request = new Request(base, {
method: 'POST',
body: new Uint8Array(stringToArrayBuffer('a=1'))
});
return request.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support DataView as body', () => {
const request = new Request(base, {
method: 'POST',
body: new DataView(stringToArrayBuffer('a=1'))
});
return request.text().then(result => {
expect(result).to.equal('a=1');
});
});
});

200
test/response.js Normal file
View File

@ -0,0 +1,200 @@
import * as stream from 'stream';
import {Response} from '../src';
import TestServer from './utils/server';
import chai from 'chai';
import resumer from 'resumer';
import stringToArrayBuffer from 'string-to-arraybuffer';
import Blob from 'fetch-blob';
const {expect} = chai;
const local = new TestServer();
const base = `http://${local.hostname}:${local.port}/`;
describe('Response', () => {
it('should have attributes conforming to Web IDL', () => {
const res = new Response();
const enumerableProperties = [];
for (const property in res) {
enumerableProperties.push(property);
}
for (const toCheck of [
'body',
'bodyUsed',
'arrayBuffer',
'blob',
'json',
'text',
'url',
'status',
'ok',
'redirected',
'statusText',
'headers',
'clone'
]) {
expect(enumerableProperties).to.contain(toCheck);
}
for (const toCheck of [
'body',
'bodyUsed',
'url',
'status',
'ok',
'redirected',
'statusText',
'headers'
]) {
expect(() => {
res[toCheck] = 'abc';
}).to.throw();
}
});
it('should support empty options', () => {
let body = resumer().queue('a=1').end();
body = body.pipe(new stream.PassThrough());
const res = new Response(body);
return res.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support parsing headers', () => {
const res = new Response(null, {
headers: {
a: '1'
}
});
expect(res.headers.get('a')).to.equal('1');
});
it('should support text() method', () => {
const res = new Response('a=1');
return res.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support json() method', () => {
const res = new Response('{"a":1}');
return res.json().then(result => {
expect(result.a).to.equal(1);
});
});
it('should support buffer() method', () => {
const res = new Response('a=1');
return res.buffer().then(result => {
expect(result.toString()).to.equal('a=1');
});
});
it('should support blob() method', () => {
const res = new Response('a=1', {
method: 'POST',
headers: {
'Content-Type': 'text/plain'
}
});
return res.blob().then(result => {
expect(result).to.be.an.instanceOf(Blob);
expect(result.size).to.equal(3);
expect(result.type).to.equal('text/plain');
});
});
it('should support clone() method', () => {
let body = resumer().queue('a=1').end();
body = body.pipe(new stream.PassThrough());
const res = new Response(body, {
headers: {
a: '1'
},
url: base,
status: 346,
statusText: 'production'
});
const cl = res.clone();
expect(cl.headers.get('a')).to.equal('1');
expect(cl.url).to.equal(base);
expect(cl.status).to.equal(346);
expect(cl.statusText).to.equal('production');
expect(cl.ok).to.be.false;
// Clone body shouldn't be the same body
expect(cl.body).to.not.equal(body);
return cl.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support stream as body', () => {
let body = resumer().queue('a=1').end();
body = body.pipe(new stream.PassThrough());
const res = new Response(body);
return res.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support string as body', () => {
const res = new Response('a=1');
return res.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support buffer as body', () => {
const res = new Response(Buffer.from('a=1'));
return res.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support ArrayBuffer as body', () => {
const res = new Response(stringToArrayBuffer('a=1'));
return res.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support blob as body', () => {
const res = new Response(new Blob(['a=1']));
return res.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support Uint8Array as body', () => {
const res = new Response(new Uint8Array(stringToArrayBuffer('a=1')));
return res.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should support DataView as body', () => {
const res = new Response(new DataView(stringToArrayBuffer('a=1')));
return res.text().then(result => {
expect(result).to.equal('a=1');
});
});
it('should default to null as body', () => {
const res = new Response();
expect(res.body).to.equal(null);
return res.text().then(result => expect(result).to.equal(''));
});
it('should default to 200 as status code', () => {
const res = new Response(null);
expect(res.status).to.equal(200);
});
it('should default to empty string as url', () => {
const res = new Response();
expect(res.url).to.equal('');
});
});

View File

@ -0,0 +1,18 @@
export default ({Assertion}, utils) => {
utils.addProperty(Assertion.prototype, 'timeout', function () {
return new Promise(resolve => {
const timer = setTimeout(() => resolve(true), 150);
this._obj.then(() => {
clearTimeout(timer);
resolve(false);
});
}).then(timeouted => {
this.assert(
timeouted,
'expected promise to timeout but it was resolved',
'expected promise not to timeout but it timed out'
);
});
});
};

View File

@ -1,24 +1,19 @@
import * as http from 'http';
import { parse } from 'url';
import * as zlib from 'zlib';
import * as stream from 'stream';
import { multipart as Multipart } from 'parted';
let convert;
try { convert = require('encoding').convert; } catch(e) {}
import {multipart as Multipart} from 'parted';
export default class TestServer {
constructor() {
this.server = http.createServer(this.router);
this.port = 30001;
this.hostname = 'localhost';
// node 8 default keepalive timeout is 5000ms
// Node 8 default keepalive timeout is 5000ms
// make it shorter here as we want to close server quickly at the end of tests
this.server.keepAliveTimeout = 1000;
this.server.on('error', function(err) {
this.server.on('error', err => {
console.log(err.stack);
});
this.server.on('connection', function(socket) {
this.server.on('connection', socket => {
socket.setTimeout(1500);
});
}
@ -31,8 +26,22 @@ export default class TestServer {
this.server.close(cb);
}
router(req, res) {
let p = parse(req.url).pathname;
mockResponse(responseHandler) {
this.server.nextResponseHandler = responseHandler;
return `http://${this.hostname}:${this.port}/mocked`;
}
router(request, res) {
const p = request.url;
if (p === '/mocked') {
if (this.nextResponseHandler) {
this.nextResponseHandler(res);
this.nextResponseHandler = undefined;
} else {
throw new Error('No mocked response. Use TestServer.mockResponse().');
}
}
if (p === '/hello') {
res.statusCode = 200;
@ -70,7 +79,11 @@ export default class TestServer {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.setHeader('Content-Encoding', 'gzip');
zlib.gzip('hello world', function(err, buffer) {
zlib.gzip('hello world', (err, buffer) => {
if (err) {
throw err;
}
res.end(buffer);
});
}
@ -79,9 +92,26 @@ export default class TestServer {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.setHeader('Content-Encoding', 'gzip');
zlib.gzip('hello world', function(err, buffer) {
// truncate the CRC checksum and size check at the end of the stream
res.end(buffer.slice(0, buffer.length - 8));
zlib.gzip('hello world', (err, buffer) => {
if (err) {
throw err;
}
// Truncate the CRC checksum and size check at the end of the stream
res.end(buffer.slice(0, -8));
});
}
if (p === '/gzip-capital') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.setHeader('Content-Encoding', 'GZip');
zlib.gzip('hello world', (err, buffer) => {
if (err) {
throw err;
}
res.end(buffer);
});
}
@ -89,7 +119,11 @@ export default class TestServer {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.setHeader('Content-Encoding', 'deflate');
zlib.deflate('hello world', function(err, buffer) {
zlib.deflate('hello world', (err, buffer) => {
if (err) {
throw err;
}
res.end(buffer);
});
}
@ -99,18 +133,25 @@ export default class TestServer {
res.setHeader('Content-Type', 'text/plain');
if (typeof zlib.createBrotliDecompress === 'function') {
res.setHeader('Content-Encoding', 'br');
zlib.brotliCompress('hello world', function (err, buffer) {
zlib.brotliCompress('hello world', (err, buffer) => {
if (err) {
throw err;
}
res.end(buffer);
});
}
}
if (p === '/deflate-raw') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.setHeader('Content-Encoding', 'deflate');
zlib.deflateRaw('hello world', function(err, buffer) {
zlib.deflateRaw('hello world', (err, buffer) => {
if (err) {
throw err;
}
res.end(buffer);
});
}
@ -130,7 +171,7 @@ export default class TestServer {
}
if (p === '/timeout') {
setTimeout(function() {
setTimeout(() => {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.end('text');
@ -141,7 +182,7 @@ export default class TestServer {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.write('test');
setTimeout(function() {
setTimeout(() => {
res.end('test');
}, 1000);
}
@ -155,10 +196,10 @@ export default class TestServer {
if (p === '/size/chunk') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
setTimeout(function() {
setTimeout(() => {
res.write('test');
}, 10);
setTimeout(function() {
setTimeout(() => {
res.end('test');
}, 20);
}
@ -169,69 +210,6 @@ export default class TestServer {
res.end('testtest');
}
if (p === '/encoding/gbk') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/html');
res.end(convert('<meta charset="gbk"><div>中文</div>', 'gbk'));
}
if (p === '/encoding/gb2312') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/html');
res.end(convert('<meta http-equiv="Content-Type" content="text/html; charset=gb2312"><div>中文</div>', 'gb2312'));
}
if (p === '/encoding/gb2312-reverse') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/html');
res.end(convert('<meta content="text/html; charset=gb2312" http-equiv="Content-Type"><div>中文</div>', 'gb2312'));
}
if (p === '/encoding/shift-jis') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/html; charset=Shift-JIS');
res.end(convert('<div>日本語</div>', 'Shift_JIS'));
}
if (p === '/encoding/euc-jp') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/xml');
res.end(convert('<?xml version="1.0" encoding="EUC-JP"?><title>日本語</title>', 'EUC-JP'));
}
if (p === '/encoding/utf8') {
res.statusCode = 200;
res.end('中文');
}
if (p === '/encoding/order1') {
res.statusCode = 200;
res.setHeader('Content-Type', 'charset=gbk; text/plain');
res.end(convert('中文', 'gbk'));
}
if (p === '/encoding/order2') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain; charset=gbk; qs=1');
res.end(convert('中文', 'gbk'));
}
if (p === '/encoding/chunked') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/html');
res.setHeader('Transfer-Encoding', 'chunked');
res.write('a'.repeat(10));
res.end(convert('<meta http-equiv="Content-Type" content="text/html; charset=Shift_JIS" /><div>日本語</div>', 'Shift_JIS'));
}
if (p === '/encoding/invalid') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/html');
res.setHeader('Transfer-Encoding', 'chunked');
res.write('a'.repeat(1200));
res.end(convert('中文', 'gbk'));
}
if (p === '/redirect/301') {
res.statusCode = 301;
res.setHeader('Location', '/inspect');
@ -276,7 +254,7 @@ export default class TestServer {
if (p === '/redirect/slow') {
res.statusCode = 301;
res.setHeader('Location', '/redirect/301');
setTimeout(function() {
setTimeout(() => {
res.end();
}, 1000);
}
@ -284,7 +262,7 @@ export default class TestServer {
if (p === '/redirect/slow-chain') {
res.statusCode = 301;
res.setHeader('Location', '/redirect/slow');
setTimeout(function() {
setTimeout(() => {
res.end();
}, 10);
}
@ -355,12 +333,14 @@ export default class TestServer {
res.statusCode = 200;
res.setHeader('Content-Type', 'application/json');
let body = '';
req.on('data', function(c) { body += c });
req.on('end', function() {
request.on('data', c => {
body += c;
});
request.on('end', () => {
res.end(JSON.stringify({
method: req.method,
url: req.url,
headers: req.headers,
method: request.method,
url: request.url,
headers: request.headers,
body
}));
});
@ -369,26 +349,32 @@ export default class TestServer {
if (p === '/multipart') {
res.statusCode = 200;
res.setHeader('Content-Type', 'application/json');
const parser = new Multipart(req.headers['content-type']);
const parser = new Multipart(request.headers['content-type']);
let body = '';
parser.on('part', function(field, part) {
parser.on('part', (field, part) => {
body += field + '=' + part;
});
parser.on('end', function() {
parser.on('end', () => {
res.end(JSON.stringify({
method: req.method,
url: req.url,
headers: req.headers,
body: body
method: request.method,
url: request.url,
headers: request.headers,
body
}));
});
req.pipe(parser);
request.pipe(parser);
}
if (p === '/m%C3%B6bius') {
res.statusCode = 200;
res.setHeader('Content-Type', 'text/plain');
res.end('ok');
}
}
}
if (require.main === module) {
const server = new TestServer;
const server = new TestServer();
server.start(() => {
console.log(`Server started listening at port ${server.port}`);
});