zec
/
simtfl
mirror of https://github.com/zcash/simtfl.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #22 from daira/bft-abstractions 

Prototype of BFT abstractions and Streamlet; add static type checking
This commit is contained in:
Daira Emma Hopwood 2023-12-07 11:32:57 +00:00 committed by GitHub
parent 54cc568a9d 76ef14abf7
commit a1641c8dc9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
19 changed files with 1187 additions and 142 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.flake8
View File

@ -2,6 +2,7 @@
exclude = .git, __pycache__
# Justifications for each ignored error or warning:
# * E252: equals for a default argument should look distinct from assignment.
# * E302: always requiring two blank lines rather than one between top-level items is too annoying and nitpicky.
# * E402: we want imports for tests to go between the main code and the tests.
# * W503, W504: these give false positives for the style of avoiding \ at the end of each line by using parens.
@ -9,6 +10,6 @@ exclude = .git, __pycache__
# References:
# - https://flake8.pycqa.org/en/latest/user/error-codes.html
# - https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes
ignore = E302, E402, W503, W504
ignore = E252, E302, E402, W503, W504
max-line-length = 120

9
.github/workflows/flake8.yml → .github/workflows/lints.yml vendored
View File

@ -1,4 +1,4 @@
name: flake8
name: lints
on: pull_request
@ -16,8 +16,15 @@ jobs:
- name: Install poetry
run: pip install --user poetry
- name: Run poetry check
run: poetry check
- name: Install dependencies
run: poetry install --no-root
- name: Run flake8
run: poetry run flake8
- name: Run pyanalyze
# `poetry run pyanalyze .` doesn't work in CI for some reason.
run: poetry run python -m pyanalyze .

14
.vscode/settings.json vendored Normal file
View File

@ -0,0 +1,14 @@
{
"python.testing.unittestArgs": [
"-s",
"./simtfl",
"-t",
".",
"-p",
"[a-z]*.py",
"--verbose",
"--buffer"
],
"python.testing.pytestEnabled": false,
"python.testing.unittestEnabled": true
}

12
README.md
View File

@ -27,7 +27,7 @@ Note the caveats: *experimental*, *simulator*, *research*, *potential*.
Design documentation is under the `doc/` directory:
* [Programming patterns for use of simpy](doc/patterns.md).
* [Programming patterns for use of simpy and type annotations](doc/patterns.md).
You can also generate API documentation by running `./gendoc.sh`. This assumes
that you have run `poetry install` as shown above. The starting point for the
@ -35,12 +35,12 @@ generated documentation is <apidoc/simtfl.html>.
## Contributing
Please use `./check.sh` before submitting a PR. This currently runs `flake8`
and the unit tests locally.
Please use `./check.sh` before submitting a PR. This currently runs `flake8`,
`pyanalyze`, and the unit tests locally.
You can use `./check.sh -k <substring>` to run `flake8` and then only tests
with names matching the given substring. This will not suppress output to
stdout or stderr (but `./check.sh -bk <substring>` will).
You can use `./check.sh -k <substring>` to run `flake8`, `pyanalyze`, and then
only tests with names matching the given substring. This will not suppress
output to stdout or stderr (but `./check.sh -bk <substring>` will).
To see other options for running unit tests, use `poetry run python -m unittest -h`.

6
check.sh
View File

@ -3,9 +3,15 @@
set -eu
cd -P -- "$(dirname -- "$(command -v -- "$0")")"
echo Running poetry check...
poetry check
echo Running flake8...
poetry run flake8
echo Running pyanalyze...
poetry run pyanalyze .
echo
echo Running unit tests...
args="${*:---buffer}"

74
doc/patterns.md
View File

@ -6,8 +6,9 @@ processes are implemented as generators, so that the library can simulate
timeouts and asynchronous communication (typically faster than real time).
We use the convention of putting "(process)" in the doc comment of these
functions. They either must use the `yield` construct, *or* return the
result of calling another "(process)" function (not both).
functions. They are also annotated with a `ProcessEffect` return type.
They either must use the `yield` construct, *or* return the result of
calling another "(process)" function (not both).
Objects that implement processes typically hold the `simpy.Environment` in
an instance variable `self.env`.
@ -18,3 +19,72 @@ statements, `return f()` can be used as an optimization.)
A "(process)" function that does nothing should `return skip()`, using
`simtfl.util.skip`.
# Type annotations
The codebase is written to use static type annotations and to pass the
[pyanalyze](https://pyanalyze.readthedocs.io/en/latest/faq.html) static
analyzer.
Each source file should have `from __future__ import annotations` at
the top. This (usually) allows types defined in the same file to be
referenced before their definition, without needing the workaround of
writing such types as string literals. The preferred style is for this
line to be immediately followed by other imports that are needed for
type annotations.
The default annotation for argument and return types is `Any`. This works
well for interoperating with libraries that don't use static typing, but
please don't rely on it for code in this project. It is better to add
explicit `Any` annotations in the few cases where that is needed. This
means that functions and methods that do not return a value should be
annotated with `-> None`.
`pyanalyze` has some limitations and is not claimed to be fully sound,
but it does a pretty good job in practice; the result feels pretty much
like a statically typed variant of Python. Importing the code it checks
allows it to be more compatible with some Python idioms. The following
workarounds for its limitations may be needed:
* It is sometimes unable to see that a `None` value cannot occur in a
particular context. In that case, adding an assertion that the value
`is not None` may help.
* In plain Python it is common to have classes that share an interface
and are used polymorphically, but have no superclass in common. It
might be necessary to create an abstract base class to define the
interface.
* There is no easy way to precisely check uses of `*args` or `**kwargs`.
* If two files have mutually dependent types, they may end up circularly
importing each other, which Python does not support. This is more
likely for types than for implementation code. There are several
possible workarounds:
* merge the files;
* move part of one file that is causing the circularity into the
other;
* create an abstract base class for the type that is being used
circularly (with methods that raise `NotImplementedError`), and
use that as the type instead of the concrete subclass.
* Adding type annotations might require exposing internal classes that
would otherwise be intentionally hidden. Since this hiding is normally
only possible by convention (e.g. using underscore-prefixed names)
in any case, that does not really cause any problem. Please just
refrain from directly using the constructors of such classes.
As is often the case for static typing in other languages, it typically
works best to use more general types for arguments and more specific
types for results.
# Flake8
We also use [flake8](https://flake8.pycqa.org/en/latest/) to encourage
a consistent coding style. However, if you disagree with a particular
error or warning produced by `flake8` and have a good justification for
why it should not be enforced, please just add it to the `ignore` list
in `.flake8`, and document the justification there.

357
poetry.lock generated
View File

@ -1,5 +1,179 @@
# This file is automatically @generated by Poetry 1.4.1 and should not be changed by hand.
[[package]]
name = "aenum"
version = "3.1.15"
description = "Advanced Enumerations (compatible with Python's stdlib Enum), NamedTuples, and NamedConstants"
category = "dev"
optional = false
python-versions = "*"
files = [
{file = "aenum-3.1.15-py2-none-any.whl", hash = "sha256:27b1710b9d084de6e2e695dab78fe9f269de924b51ae2850170ee7e1ca6288a5"},
{file = "aenum-3.1.15-py3-none-any.whl", hash = "sha256:e0dfaeea4c2bd362144b87377e2c61d91958c5ed0b4daf89cb6f45ae23af6288"},
{file = "aenum-3.1.15.tar.gz", hash = "sha256:8cbd76cd18c4f870ff39b24284d3ea028fbe8731a58df3aa581e434c575b9559"},
]
[[package]]
name = "ast-decompiler"
version = "0.7.0"
description = "Python module to decompile AST to Python code"
category = "dev"
optional = false
python-versions = ">=3.6"
files = [
{file = "ast_decompiler-0.7.0-py3-none-any.whl", hash = "sha256:5ebd37ba129227484daff4a15dd6056d87c488fa372036dd004ee84196b207d3"},
{file = "ast_decompiler-0.7.0.tar.gz", hash = "sha256:efc3a507e5f8963ec7b4b2ce2ea693e3755c2f52b741c231bc344a4526738337"},
]
[[package]]
name = "asynq"
version = "1.5.1"
description = "Quora's asynq library"
category = "dev"
optional = false
python-versions = "*"
files = [
{file = "asynq-1.5.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:9b32dc05ee205901d4d1bc136fd69652b590b79e9f34c7d371ae4d804bb13443"},
{file = "asynq-1.5.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:54bbb27169d274047646aa1ef5179d445039e5da69b972059c4379cb63fbf99d"},
{file = "asynq-1.5.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f2a809aa5d5968f45900599edbc3de6a74a3312f7c6e02e9fee6704b76675643"},
{file = "asynq-1.5.1-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:21930da219677de6ae8a648f5bb2e6fca22bf111337c0f8cccee2c199bee5273"},
{file = "asynq-1.5.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:bea141057e23a8a9ec382832983c4d72b08eb9cac613d63481af8731725e7d7b"},
{file = "asynq-1.5.1-cp310-cp310-win32.whl", hash = "sha256:315481be6b403b19e355ed28c3c6c0b47afc9c4d1fa30a8b4e2d6e1da573a6a7"},
{file = "asynq-1.5.1-cp310-cp310-win_amd64.whl", hash = "sha256:94e854a237d07b1892b851003439534a7ae6c8f640c78d19101ee30ea98bf0f9"},
{file = "asynq-1.5.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:7aaf135693d2db7db034d265616ecbe627df2a7ceac58a0be111f488acd72df9"},
{file = "asynq-1.5.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b63aef73898af7ee2d67397e2fddf723d8b114772f9f144453d1646394be0b43"},
{file = "asynq-1.5.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:58766ffb80bd2544e935e6f1a47e6cc281003d71ae2eed2d656da60f87296f48"},
{file = "asynq-1.5.1-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:66e2e1b283957c2bf21b98e9e8d6b1efd97821711bb4054025a3713a4dd4b576"},
{file = "asynq-1.5.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:3d699ca115f66731f8378bcae28309349aa6374d3379ac53a9b48e59f2ad5320"},
{file = "asynq-1.5.1-cp311-cp311-win32.whl", hash = "sha256:909d0e9395a15248cc5465ebb06efaecd3c1101fb9332e78e9f71ba1b2d28fdd"},
{file = "asynq-1.5.1-cp311-cp311-win_amd64.whl", hash = "sha256:2d24b0230ee7cd565ef79a0a1b53a8f3fd95c2cf28cdb908e733faa160511075"},
{file = "asynq-1.5.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:4c2cce53ce7719a298d9631cd077a69776fc7e91fd6e09a93817c03153f0506b"},
{file = "asynq-1.5.1-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f8e3aef44474f7f011bac97e1a92e255f201fc588464fb5f42513bc1de311b07"},
{file = "asynq-1.5.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8ab7e4903499304477b43272809c5512b39551b5794df473038aa0cd3139d30c"},
{file = "asynq-1.5.1-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:ef4baaa71d0e9c7b89b6f8282df06193d3846770ba8d2b042cb3f2ed8aa94d8f"},
{file = "asynq-1.5.1-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:ccfa6bc026e286fd5eb07ca18ca0afee4d0c09c726605f66fa32102bc9c85ecb"},
{file = "asynq-1.5.1-cp36-cp36m-win32.whl", hash = "sha256:5d75b7f07d54b3a72a196309215023fece4aa810ed7a2acaa8defd9e5011e3e1"},
{file = "asynq-1.5.1-cp36-cp36m-win_amd64.whl", hash = "sha256:210365fef54fc7a93f06bdd9ae4e7029537aa3528f4e3d00ea77b4430961e208"},
{file = "asynq-1.5.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:819842445c8ca418d303ae31bcdca7adb83afd54ea823a358306c376395ce06d"},
{file = "asynq-1.5.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f20994fc3b3ed762d8c62197b508d8a437f7f535fef395e9f33748a6b2bf8535"},
{file = "asynq-1.5.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a9592611b3317561617d51b41f45a745d64eefe2324f9380c8cff78100b94521"},
{file = "asynq-1.5.1-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:6a5843014e7b59a02cd20ced5e777870fab593d1107a145cb176540fc2c4ad56"},
{file = "asynq-1.5.1-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:7bcd6b10ae015c70a37cede9bd5dec05c30468552760c336ccef4f430c47e392"},
{file = "asynq-1.5.1-cp37-cp37m-win32.whl", hash = "sha256:487fba1aa6179d9af29f4ffe5e8f99044432e318d95501bb7cd7e406e1615df1"},
{file = "asynq-1.5.1-cp37-cp37m-win_amd64.whl", hash = "sha256:07f3e46d71de0c6379d4ec73d482cac30ac2ff42ba942089272c0512f87e0470"},
{file = "asynq-1.5.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:45507c91514a73127ef904ee81672c4ad869e865c84abe9d751ba06fa8af8915"},
{file = "asynq-1.5.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:775e8cc8bd2046a256f688641de3b93799dfcbc26bf454929ca47f3bfbc2aa85"},
{file = "asynq-1.5.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:89bc8050755c61bef6a01db0ba3dc3ec80978f52bb4b05e8a2ba214e8ecb9520"},
{file = "asynq-1.5.1-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:bf2fe2af6978e4c57a54838fb7d64d8051edc9c4154101d4b62106974cedb96f"},
{file = "asynq-1.5.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:839500d3b504ff43b0d153526fb726682384ab8bada6fd84ae462d63170bb527"},
{file = "asynq-1.5.1-cp38-cp38-win32.whl", hash = "sha256:c5c139dbfa4cdecb411fd7298ec275fd6ce0e6845ef91dfb213162ed9f761694"},
{file = "asynq-1.5.1-cp38-cp38-win_amd64.whl", hash = "sha256:80f3d4d2cc9d5453f7d57570ae8a186e2cd491f76acc9085bcc2008511f72c0e"},
{file = "asynq-1.5.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:cf848faf162ad655058ae0e99929750fc75d59d390e6b96dabd482a4ef4b7798"},
{file = "asynq-1.5.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:12ec21dee0027d9783c3d44641304e72ac3928fb293e227874cfdab76cec74b1"},
{file = "asynq-1.5.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:089833f67f178f93cfbc3ced48300591197b8afd51b83bb2e27c97b3689a57a6"},
{file = "asynq-1.5.1-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:d38ea10eb99c60bcccfbc6b1bd88a3edf7ff7db98a1cb3a166e181f7d0650964"},
{file = "asynq-1.5.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:7b5431242f09e7b6da47d0df1c56b6533d49acc1c84f23f4adaf0424f12e5295"},
{file = "asynq-1.5.1-cp39-cp39-win32.whl", hash = "sha256:040e1d02fce358b1622cef41ba1f1e07bacce7736d4419c9e4ae096d5112638a"},
{file = "asynq-1.5.1-cp39-cp39-win_amd64.whl", hash = "sha256:0f1146c2a91fc7cbfdc73036fcd4aa434121c04619509747d6ef952b669c540b"},
{file = "asynq-1.5.1-pp37-pypy37_pp73-macosx_10_9_x86_64.whl", hash = "sha256:3210c05349d5582b51b23da69b3e5d1a5d11256782eff318327ebebb95d745ca"},
{file = "asynq-1.5.1-pp37-pypy37_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:eb98e69916bc2c3e27f25634c4734a481a56778e6cd2bd176861fcb5dc38a988"},
{file = "asynq-1.5.1-pp37-pypy37_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7880e44d7daef4eace2168afd763c95950bab3e54150ba803948939e2c927a25"},
{file = "asynq-1.5.1-pp37-pypy37_pp73-win_amd64.whl", hash = "sha256:46b3b262c0f60901a4ce23daf4ba82b024cdff547e04fdcb00719a95b01a8c37"},
{file = "asynq-1.5.1-pp38-pypy38_pp73-macosx_10_9_x86_64.whl", hash = "sha256:779a66c9440fe2e0d709b37406fe3977ecda66ae41f880b8cf34f79e0df1b451"},
{file = "asynq-1.5.1-pp38-pypy38_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c9733a90510fddeb31976d3dfa449721ca8bd9caaf6d3d5fb0bc04032b094aa4"},
{file = "asynq-1.5.1-pp38-pypy38_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:53dd5b7ea6ce911a3967485bbc8e3c435dc1008e9f31ea97f57804617828eea7"},
{file = "asynq-1.5.1-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:40fefa839fe2cfbcd072c7249c0c4149af75d1d48573f3fc1e01c6b2bb6169f1"},
{file = "asynq-1.5.1-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:a7be45e9336f52a9329cde19ece904fd93611da3602f5a98fbf119406dc6f340"},
{file = "asynq-1.5.1-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6d5bbd5b68daf3e1a669c540cdc7a91a2ac9c49729ffae06d1790358759a1025"},
{file = "asynq-1.5.1-pp39-pypy39_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2cc72527d002fab4bca0d560e82e440ae69d42e393c59747ad2c8a2eb972aaa8"},
{file = "asynq-1.5.1-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:8774eb6a042b1a011f1f7c3db9f555e83929184139fe2bfa1490a893819b04e5"},
{file = "asynq-1.5.1.tar.gz", hash = "sha256:d1ae6a9b4934e821e22a08369512756d0edb9339436ae805b5fc815dacc4421e"},
]
[package.dependencies]
Cython = ">=0.27.1"
pygments = "*"
qcore = "*"
[[package]]
name = "codemod"
version = "1.0.0"
description = "Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention. Codemod was developed at Facebook and released as open source."
category = "dev"
optional = false
python-versions = "*"
files = [
{file = "codemod-1.0.0.tar.gz", hash = "sha256:06e8c75f2b45210dd8270e30a6a88ae464b39abd6d0cab58a3d7bfd1c094e588"},
]
[[package]]
name = "cython"
version = "3.0.6"
description = "The Cython compiler for writing C extensions in the Python language."
category = "dev"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
files = [
{file = "Cython-3.0.6-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:0fcdfbf6fc7d0bd683d55e617c3d5a5f25b28ce8b405bc1e89054fc7c52a97e5"},
{file = "Cython-3.0.6-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ccbee314f8d15ee8ddbe270859dda427e1187123f2c7c41526d1f260eee6c8f7"},
{file = "Cython-3.0.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:14b992f36ffa1294921fca5f6488ea192fadd75770dc64fa25975379382551e9"},
{file = "Cython-3.0.6-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2ca2e90a75d405070f3c41e701bb8005892f14d42322f1d8fd00a61d660bbae7"},
{file = "Cython-3.0.6-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:4121c1160bc1bd8828546e8ce45906bd9ff27799d14747ce3fbbc9d67efbb1b8"},
{file = "Cython-3.0.6-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:519814b8f80869ee5f9ee2cb2363e5c310067c0298cbea291c556b22da1ef6ae"},
{file = "Cython-3.0.6-cp310-cp310-win32.whl", hash = "sha256:b029d8c754ef867ab4d67fc2477dde9782bf0409cb8e4024a7d29cf5aff37530"},
{file = "Cython-3.0.6-cp310-cp310-win_amd64.whl", hash = "sha256:2262390f453eedf600e084b074144286576ed2a56bb7fbfe15ad8d9499eceb52"},
{file = "Cython-3.0.6-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:dfe8c7ac60363769ed8d91fca26398aaa9640368ab999a79b0ccb5e788d3bcf8"},
{file = "Cython-3.0.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9e31a9b18ec6ce57eb3479df920e6093596fe4ba8010dcc372720040386b4bdb"},
{file = "Cython-3.0.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bca2542f1f34f0141475b13777df040c31f2073a055097734a0a793ac3a4fb72"},
{file = "Cython-3.0.6-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b24c1c38dad4bd85e142ccbe2f88122807f8d5a75352321e1e4baf2b293df7c6"},
{file = "Cython-3.0.6-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:dc4b4e76c1414584bb55465dfb6f41dd6bd27fd53fb41ddfcaca9edf00c1f80e"},
{file = "Cython-3.0.6-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:805a2c532feee09aeed064eaeb7b6ee35cbab650569d0a3756975f3cc4f246cf"},
{file = "Cython-3.0.6-cp311-cp311-win32.whl", hash = "sha256:dcdb9a177c7c385fe0c0709a9a6790b6508847d67dcac76bb65a2c7ea447efe5"},
{file = "Cython-3.0.6-cp311-cp311-win_amd64.whl", hash = "sha256:b8640b7f6503292c358cef925df5a69adf230045719893ffe20ad98024fdf7ae"},
{file = "Cython-3.0.6-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:16b3b02cc7b3bc42ee1a0118b1465ca46b0f3fb32d003e6f1a3a352a819bb9a3"},
{file = "Cython-3.0.6-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:11e1d9b153573c425846b627bef52b3b99cb73d4fbfbb136e500a878d4b5e803"},
{file = "Cython-3.0.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:85a7a406f78c2f297bf82136ff5deac3150288446005ed1e56552a9e3ac1469f"},
{file = "Cython-3.0.6-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:88be4fbc760de8f313df89ca8256098c0963c9ec72f3aa88538384b80ef1a6ef"},
{file = "Cython-3.0.6-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:ea2e5a7c503b41618bfb10e4bc610f780ab1c729280531b5cabb24e05aa21cf2"},
{file = "Cython-3.0.6-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:2d296b48e1410cab50220a28a834167f2d7ac6c0e7de12834d66e42248a1b0f6"},
{file = "Cython-3.0.6-cp312-cp312-win32.whl", hash = "sha256:7f19e99c6e334e9e30dfa844c3ca4ac09931b94dbba406c646bde54687aed758"},
{file = "Cython-3.0.6-cp312-cp312-win_amd64.whl", hash = "sha256:9cae02e26967ffb6503c6e91b77010acbadfb7189a5a11d6158d634fb0f73679"},
{file = "Cython-3.0.6-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:cb6a54543869a5b0ad009d86eb0ebc0879fab838392bfd253ad6d4f5e0f17d84"},
{file = "Cython-3.0.6-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8d2d9e53bf021cc7a5c7b6b537b5b5a7ba466ba7348d498aa17499d0ad12637e"},
{file = "Cython-3.0.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:05d15854b2b363b35c755d22015c1c2fc590b8128202f8c9eb85578461101d9c"},
{file = "Cython-3.0.6-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e5548316497a3b8b2d9da575ea143476472db90dee73c67def061621940f78ae"},
{file = "Cython-3.0.6-cp36-cp36m-musllinux_1_1_aarch64.whl", hash = "sha256:9b853e0855e4b3d164c05b24718e5e2df369e5af54f47cb8d923c4f497dfc92c"},
{file = "Cython-3.0.6-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:2c77f97f462a40a319dda7e28c1669370cb26f9175f3e8f9bab99d2f8f3f2f09"},
{file = "Cython-3.0.6-cp36-cp36m-win32.whl", hash = "sha256:3ac8b6734f2cad5640f2da21cd33cf88323547d07e445fb7453ab38ec5033b1f"},
{file = "Cython-3.0.6-cp36-cp36m-win_amd64.whl", hash = "sha256:8dd5f5f3587909ff71f0562f50e00d4b836c948e56e8f74897b12f38a29e41b9"},
{file = "Cython-3.0.6-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:9c0472c6394750469062deb2c166125b10411636f63a0418b5c36a60d0c9a96a"},
{file = "Cython-3.0.6-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:97081932c8810bb99cb26b4b0402202a1764b58ee287c8b306071d2848148c24"},
{file = "Cython-3.0.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e781b3880dfd0d4d37983c9d414bfd5f26c2141f6d763d20ef1964a0a4cb2405"},
{file = "Cython-3.0.6-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ef88c46e91e21772a5d3b6b1e70a6da5fe098154ad4768888129b1c05e93bba7"},
{file = "Cython-3.0.6-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:a38b9e7a252ec27dbc21ee8f00f09a896e88285eebb6ed99207b2ff1ea6af28e"},
{file = "Cython-3.0.6-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:4975cdaf720d29288ec225b76b4f4471ff03f4f8b51841ba85d6587699ab2ad5"},
{file = "Cython-3.0.6-cp37-cp37m-win32.whl", hash = "sha256:9b89463ea330318461ca47d3e49b5f606e7e82446b6f37e5c19b60392439674c"},
{file = "Cython-3.0.6-cp37-cp37m-win_amd64.whl", hash = "sha256:0ca8f379b47417bfad98faeb14bf8a3966fc92cf69f8aaf7635cf6885e50d001"},
{file = "Cython-3.0.6-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:b3dda1e80eb577b9563cee6cf31923a7b88836b9f9be0043ec545b138b95d8e8"},
{file = "Cython-3.0.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:267e34e9a96f98c379100ef4192994a311678fb5c9af34c83ba5230223577581"},
{file = "Cython-3.0.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:345d9112fde4ae0347d656f58591fd52017c61a19779c95423bb38735fe4a401"},
{file = "Cython-3.0.6-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:25da0e51331ac12ff16cd858d1d836e092c984e1dc45d338166081d3802297c0"},
{file = "Cython-3.0.6-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:eebbf09089b4988b9f398ed46f168892e32fcfeec346b15954fdd818aa103456"},
{file = "Cython-3.0.6-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:e3ed0c125556324fa49b9e92bea13be7b158fcae6f72599d63c8733688257788"},
{file = "Cython-3.0.6-cp38-cp38-win32.whl", hash = "sha256:86e1e5a5c9157a547d0a769de59c98a1fc5e46cfad976f32f60423cc6de11052"},
{file = "Cython-3.0.6-cp38-cp38-win_amd64.whl", hash = "sha256:0d45a84a315bd84d1515cd3571415a0ee0709eb4e2cd4b13668ede928af344a7"},
{file = "Cython-3.0.6-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:a8e788e64b659bb8fe980bc37da3118e1f7285dec40c5fb293adabc74d4205f2"},
{file = "Cython-3.0.6-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9a77a174c7fb13d80754c8bf9912efd3f3696d13285b2f568eca17324263b3f7"},
{file = "Cython-3.0.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1074e84752cd0daf3226823ddbc37cca8bc45f61c94a1db2a34e641f2b9b0797"},
{file = "Cython-3.0.6-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:49d5cae02d56e151e1481e614a1af9a0fe659358f2aa5eca7a18f05aa641db61"},
{file = "Cython-3.0.6-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:0b94610fa49e36db068446cfd149a42e3246f38a4256bbe818512ac181446b4b"},
{file = "Cython-3.0.6-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:fabb2d14dd71add618a7892c40ffec584d1dae1e477caa193778e52e06821d83"},
{file = "Cython-3.0.6-cp39-cp39-win32.whl", hash = "sha256:ce442c0be72ab014c305399d955b78c3d1e69d5a5ce24398122b605691b69078"},
{file = "Cython-3.0.6-cp39-cp39-win_amd64.whl", hash = "sha256:8a05f79a0761fc76c42e945e5a9cb5d7986aa9e8e526fdf52bd9ca61a12d4567"},
{file = "Cython-3.0.6-py2.py3-none-any.whl", hash = "sha256:5921a175ea20779d4443ef99276cfa9a1a47de0e32d593be7679be741c9ed93b"},
{file = "Cython-3.0.6.tar.gz", hash = "sha256:399d185672c667b26eabbdca420c98564583798af3bc47670a8a09e9f19dd660"},
]
[[package]]
name = "flake8"
version = "6.1.0"
@ -17,6 +191,25 @@ mccabe = ">=0.7.0,<0.8.0"
pycodestyle = ">=2.11.0,<2.12.0"
pyflakes = ">=3.1.0,<3.2.0"
[[package]]
name = "importlib-resources"
version = "6.1.1"
description = "Read resources from Python packages"
category = "dev"
optional = false
python-versions = ">=3.8"
files = [
{file = "importlib_resources-6.1.1-py3-none-any.whl", hash = "sha256:e8bf90d8213b486f428c9c39714b920041cb02c184686a3dee24905aaa8105d6"},
{file = "importlib_resources-6.1.1.tar.gz", hash = "sha256:3893a00122eafde6894c59914446a512f728a0c1a45f9bb9b63721b6bacf0b4a"},
]
[package.dependencies]
zipp = {version = ">=3.1.0", markers = "python_version < \"3.10\""}
[package.extras]
docs = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linker (>=1.9)", "sphinx (<7.2.5)", "sphinx (>=3.5)", "sphinx-lint"]
testing = ["pytest (>=6)", "pytest-black (>=0.3.7)", "pytest-checkdocs (>=2.4)", "pytest-cov", "pytest-enabler (>=2.2)", "pytest-mypy (>=0.9.1)", "pytest-ruff", "zipp (>=3.17)"]
[[package]]
name = "jinja2"
version = "3.1.2"
@ -137,6 +330,31 @@ pygments = ">=2.12.0"
[package.extras]
dev = ["black", "hypothesis", "mypy", "pygments (>=2.14.0)", "pytest", "pytest-cov", "pytest-timeout", "ruff", "tox", "types-pygments"]
[[package]]
name = "pyanalyze"
version = "0.11.0"
description = "A static analyzer for Python"
category = "dev"
optional = false
python-versions = ">=3.8"
files = [
{file = "pyanalyze-0.11.0-py3-none-any.whl", hash = "sha256:f2e5c1023eca53f7825c64ef44721f667810f8f086f817df22711337bb9ab683"},
{file = "pyanalyze-0.11.0.tar.gz", hash = "sha256:0258648e2c919f849343cc942c61556fff189b03dea563f2b4c48620c4ca4ab5"},
]
[package.dependencies]
aenum = ">=2.2.3"
ast-decompiler = ">=0.4.0"
asynq = "*"
codemod = "*"
qcore = ">=0.5.1"
tomli = ">=1.1.0"
typeshed-client = ">=2.1.0"
typing-extensions = ">=4.1.0"
[package.extras]
tests = ["annotated-types", "attrs", "mypy-extensions", "pydantic", "pytest"]
[[package]]
name = "pycodestyle"
version = "2.11.1"
@ -163,32 +381,153 @@ files = [
[[package]]
name = "pygments"
version = "2.16.1"
version = "2.17.2"
description = "Pygments is a syntax highlighting package written in Python."
category = "dev"
optional = false
python-versions = ">=3.7"
files = [
{file = "Pygments-2.16.1-py3-none-any.whl", hash = "sha256:13fc09fa63bc8d8671a6d247e1eb303c4b343eaee81d861f3404db2935653692"},
{file = "Pygments-2.16.1.tar.gz", hash = "sha256:1daff0494820c69bc8941e407aa20f577374ee88364ee10a98fdbe0aece96e29"},
{file = "pygments-2.17.2-py3-none-any.whl", hash = "sha256:b27c2826c47d0f3219f29554824c30c5e8945175d888647acd804ddd04af846c"},
{file = "pygments-2.17.2.tar.gz", hash = "sha256:da46cec9fd2de5be3a8a784f434e4c4ab670b4ff54d605c4c2717e9d49c4c367"},
]
[package.extras]
plugins = ["importlib-metadata"]
windows-terminal = ["colorama (>=0.4.6)"]
[[package]]
name = "qcore"
version = "1.10.0"
description = "Quora's core utility library"
category = "dev"
optional = false
python-versions = "*"
files = [
{file = "qcore-1.10.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:230d5139ed85ee7befe1291e36cd9e9f83b14437fad34c01ca08a51fc5b4e07f"},
{file = "qcore-1.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:658469d6150d9be77812f2f5c1d3333a1c59b744acb8072172a44d2bc876ec89"},
{file = "qcore-1.10.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:3c3d8477c17153326571fcc60e74b0f337fda4e8c0a72e4347e0a82a22c70211"},
{file = "qcore-1.10.0-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:9648a93350023dc514a04a94a7a48d32683493e8d7481cca4803d3773d3d3d49"},
{file = "qcore-1.10.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:ae8accf69ed503fb41e76a3e5846b0e3f41d6e0a374c2891ea46a57958039439"},
{file = "qcore-1.10.0-cp310-cp310-win32.whl", hash = "sha256:77f4beeb717aec0ad90c8b1f9e30d8cb599d682d1f3ac574707ffa54709d3942"},
{file = "qcore-1.10.0-cp310-cp310-win_amd64.whl", hash = "sha256:d2bfd706ecf78763580e46911a7b01e32741d3f51eb52e51a12235120573c3a6"},
{file = "qcore-1.10.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:aad41cf2c751d96720b026585427e4764955320f72c581ef270e6415f2284230"},
{file = "qcore-1.10.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f369f4761e1c171d91c721701225219776440ae93c0f0bf2a65d2ab774ff7070"},
{file = "qcore-1.10.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:675ef17fef24b2853c169cdb3c5da00aa93d7f6dd3590b6c5a48fd6984d51192"},
{file = "qcore-1.10.0-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:0d1b3ca6cc5aa9d413814e088102a2f5680b89da30dada45d2d854fb78a00833"},
{file = "qcore-1.10.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:6da2119af6a5d59a304cf2dfb9beaa450e54aa79b239840ea4047ecae6659cbb"},
{file = "qcore-1.10.0-cp311-cp311-win32.whl", hash = "sha256:6a7fcfabc2be07d728f0f507bba8bc79df0895f0ecf2f19f7b0f70928318880a"},
{file = "qcore-1.10.0-cp311-cp311-win_amd64.whl", hash = "sha256:a3e317e0406edb0c6b776d37dc5c6891cd50ec9adda285b379cd2098a6dbb6de"},
{file = "qcore-1.10.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:cefb20a1ab5ca5c34a86dc2b4f52f12cc909ba574912aef13b2f738912429a9d"},
{file = "qcore-1.10.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1b003fb8116fd815a6258e79b0eb5d77743faabd4c51d0142904241bb97b7115"},
{file = "qcore-1.10.0-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b290c99d1fa4c103b27de2c631db57d705eb0c7bf935158a5335b9529604951b"},
{file = "qcore-1.10.0-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:abc664b95281a671b0fddabbd875d449d1ec616a05fc71d53caed8f2130ebc02"},
{file = "qcore-1.10.0-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:f948a5a3064bf7d8f59e94b891e519718d01bafc0e9f3850017cb4b935d21040"},
{file = "qcore-1.10.0-cp36-cp36m-win32.whl", hash = "sha256:34d55e604ee02ec1235248dca427d88ac06ccc28fe3cba618cceaf0d4fac595c"},
{file = "qcore-1.10.0-cp36-cp36m-win_amd64.whl", hash = "sha256:3b30b6ffe2067e21c882ed3093538f0361dc800de63e0e15a099cf71c08dfd62"},
{file = "qcore-1.10.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:123b0605634155f7fb3fd88b0120d7f05c2dbe714cbbf3097eaa59f6e28f3bbc"},
{file = "qcore-1.10.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:23e0e8173330b5433586ba6362a57c3424ae90add7a967c5ab11a3768492873f"},
{file = "qcore-1.10.0-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:02d7d6f3c9583403a16d0bd1905dc3ccb3edbca3529b1c2da8b09ef17599aa00"},
{file = "qcore-1.10.0-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:44c2309531e67b9b9531eb9318940a8cb6cd9a4bca1b0ae01f79925527c77790"},
{file = "qcore-1.10.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:8ef47b985a75cabbac656b38643b6c17114359b9a290cdf0e7e5bd9031b7672e"},
{file = "qcore-1.10.0-cp37-cp37m-win32.whl", hash = "sha256:91d742174f7beff7b967ddf80253fa95f055a2063d69c73d40d4d928cec3f72a"},
{file = "qcore-1.10.0-cp37-cp37m-win_amd64.whl", hash = "sha256:21b2952625a781465e67789fe687819d704800fd32ad19493f3225031869a0a0"},
{file = "qcore-1.10.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:044008dbe127bf2d17a618210d607c73a0894a887185ff871b9d45f755469a6b"},
{file = "qcore-1.10.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:071e933bbabea344ae9062af892f60351a5eec30f8f16230aa334bd17d82f09e"},
{file = "qcore-1.10.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8037cc36d2e64590a9974375e88047225dc8f00297d40cf581e38a765b9cd88d"},
{file = "qcore-1.10.0-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:79402feea121bdae1ba595e199c681c6bbcef19211a79cfcad744519b872c44f"},
{file = "qcore-1.10.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:0fe05dbc40483cf53c348faff7e1463c4c7465907072342c330c9663f1435dd7"},
{file = "qcore-1.10.0-cp38-cp38-win32.whl", hash = "sha256:d96feb04f6521bde1eae9ae6c4b62bed47e9c84adb8c28b1dbc592e8bfa5f5ce"},
{file = "qcore-1.10.0-cp38-cp38-win_amd64.whl", hash = "sha256:65dc62d408f340b9bb0a3d0dd2a506218dd614439a63898c1777c0b0000b517e"},
{file = "qcore-1.10.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:c0bf625f48d74f1e4f4550df5e79626a60051ed2e3ec413a7eeec91a35225636"},
{file = "qcore-1.10.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1f347126bdc32d8c571a390df579c1abbea01c210f8a8c3d37d956ba01e53c60"},
{file = "qcore-1.10.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:511fa1a71bbb359b4105a4afeb577a043681b5cb93d877909421e9f7a3c595b2"},
{file = "qcore-1.10.0-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:d45dddc8b02f4d968b034438b7be08ca95dc799465d5fdfc6d5a525ebaf96aeb"},
{file = "qcore-1.10.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:df87f1a2c8e69fe9b916047383949919a445574b68400f59c888a4456df1110d"},
{file = "qcore-1.10.0-cp39-cp39-win32.whl", hash = "sha256:48ffdc18f930a34d22c466fd0c6ce1832848c6d82f77caab13d5c6788ca348b2"},
{file = "qcore-1.10.0-cp39-cp39-win_amd64.whl", hash = "sha256:566b1d3bd6771c7db9ba7188ab3daa1e9cf9290d8d52f20e7132e1c2d95979fd"},
{file = "qcore-1.10.0-pp37-pypy37_pp73-macosx_10_9_x86_64.whl", hash = "sha256:50cc977da46c5dad7a98b3983a62e19100cec8043e8c24ff7e1a3f83630ed970"},
{file = "qcore-1.10.0-pp37-pypy37_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:37b264a0b83f513d4fd1b12bbb8339146ae1893d18f9476e3df562f50620d2bf"},
{file = "qcore-1.10.0-pp37-pypy37_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:72555913df64b0d3487f5d3156d7a2a618eb19f91e67ff50e9237915d8b6ed0b"},
{file = "qcore-1.10.0-pp37-pypy37_pp73-win_amd64.whl", hash = "sha256:3e619573e2f0b8e16cd1f43ff4801d6cab541031be48ae1711c645445ea11755"},
{file = "qcore-1.10.0-pp38-pypy38_pp73-macosx_10_9_x86_64.whl", hash = "sha256:68d5aec5186799fcbe651c383b489c27ba9c23624ea293ce7904fab36c0fe8c8"},
{file = "qcore-1.10.0-pp38-pypy38_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:cfbb183a804f986d37a3f035dfd53d536271d020b4636e85ee7cfa30ca7a83e9"},
{file = "qcore-1.10.0-pp38-pypy38_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:97e564ae11157c88708042aaf3a615bafcde258b6f18832f1605ef5297fd6d5b"},
{file = "qcore-1.10.0-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:8b1737f71952a1f184d946200af0f50b93339228fd19e4f995693b003d7ad3fc"},
{file = "qcore-1.10.0-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:fd756b471b74009bf4d403aa7f0c97a82a14a50288cfa9b547acb3563b84790d"},
{file = "qcore-1.10.0-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:dd729c2e01d0da7acc77f32f471cbe2d0777b883419546055bf8d579b5b0a73d"},
{file = "qcore-1.10.0-pp39-pypy39_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ff5fa19dffac882e564a9e2e0314c1b5373747b9aad0054f3570673f7e8f71ed"},
{file = "qcore-1.10.0-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:0fbc0d69a3b973641e520fc9d139753dad971978a13361d808e878499eeea010"},
{file = "qcore-1.10.0.tar.gz", hash = "sha256:88c3dcc0f1b4843eae063334fb4b278c4660c36db6382ffa98bc1150be71e8a1"},
]
[[package]]
name = "simpy"
version = "4.0.2"
version = "4.1.1"
description = "Event discrete, process based simulation for Python."
category = "main"
optional = false
python-versions = ">=3.6"
python-versions = ">=3.8"
files = [
{file = "simpy-4.0.2-py2.py3-none-any.whl", hash = "sha256:603cdf4299e396c9f16b10806e749decb0d08a7e72e0c26f9eb9762b9bde29cc"},
{file = "simpy-4.0.2.tar.gz", hash = "sha256:6d8adc0229df6b02fb7e26dcd1338703b4f4f63f167a5ac2a7213cb80aba4484"},
{file = "simpy-4.1.1-py3-none-any.whl", hash = "sha256:7c5ae380240fd2238671160e4830956f8055830a8317edf5c05e495b3823cd88"},
{file = "simpy-4.1.1.tar.gz", hash = "sha256:06d0750a7884b11e0e8e20ce0bc7c6d4ed5f1743d456695340d13fdff95001a6"},
]
[[package]]
name = "tomli"
version = "2.0.1"
description = "A lil' TOML parser"
category = "dev"
optional = false
python-versions = ">=3.7"
files = [
{file = "tomli-2.0.1-py3-none-any.whl", hash = "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc"},
{file = "tomli-2.0.1.tar.gz", hash = "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"},
]
[[package]]
name = "typeshed-client"
version = "2.4.0"
description = "A library for accessing stubs in typeshed."
category = "dev"
optional = false
python-versions = ">=3.7"
files = [
{file = "typeshed_client-2.4.0-py3-none-any.whl", hash = "sha256:5358cab27cf2d7b1cd1e77dd92a3ac3cd9cd31df9eb2e958bd280a38160a3219"},
{file = "typeshed_client-2.4.0.tar.gz", hash = "sha256:b4e4e3e40dca91ce1a667d2eb0eb350a0a2c0d80e18a232d18857aa61bed3492"},
]
[package.dependencies]
importlib-resources = ">=1.4.0"
[[package]]
name = "typing-extensions"
version = "4.8.0"
description = "Backported and Experimental Type Hints for Python 3.8+"
category = "dev"
optional = false
python-versions = ">=3.8"
files = [
{file = "typing_extensions-4.8.0-py3-none-any.whl", hash = "sha256:8f92fc8806f9a6b641eaa5318da32b44d401efaac0f6678c9bc448ba3605faa0"},
{file = "typing_extensions-4.8.0.tar.gz", hash = "sha256:df8e4339e9cb77357558cbdbceca33c303714cf861d1eef15e1070055ae8b7ef"},
]
[[package]]
name = "zipp"
version = "3.17.0"
description = "Backport of pathlib-compatible object wrapper for zip files"
category = "dev"
optional = false
python-versions = ">=3.8"
files = [
{file = "zipp-3.17.0-py3-none-any.whl", hash = "sha256:0e923e726174922dce09c53c59ad483ff7bbb8e572e00c7f7c46b88556409f31"},
{file = "zipp-3.17.0.tar.gz", hash = "sha256:84e64a1c28cf7e91ed2078bb8cc8c259cb19b76942096c8d7b84947690cabaf0"},
]
[package.extras]
docs = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linker (>=1.9)", "sphinx (<7.2.5)", "sphinx (>=3.5)", "sphinx-lint"]
testing = ["big-O", "jaraco.functools", "jaraco.itertools", "more-itertools", "pytest (>=6)", "pytest-black (>=0.3.7)", "pytest-checkdocs (>=2.4)", "pytest-cov", "pytest-enabler (>=2.2)", "pytest-ignore-flaky", "pytest-mypy (>=0.9.1)", "pytest-ruff"]
[metadata]
lock-version = "2.0"
python-versions = "^3.10"
content-hash = "d9f1f22d9f90f2ad4d00ec2785eee468e847dc1c8c4015615febe6094edf3e62"
python-versions = "^3.9.2"
content-hash = "7a2dfd4ab1e5ffb3daf67698df7cb1c9969cf7963b9c74d9c1b6b31bc9e01f6d"

2
pyproject.toml
View File

@ -13,10 +13,12 @@ simpy = "^4"
[tool.poetry.dev-dependencies]
flake8 = "^6"
pdoc = "^14"
pyanalyze = "^0.11"
[tool.poetry.scripts]
demo = "simtfl.demo:run"
bc-demo = "simtfl.bc.demo:run"
pyanalyze = "pyanalyze.__main__:main"
[build-system]
requires = ["poetry-core"]

73
simtfl/bc/__init__.py
View File

@ -15,10 +15,15 @@ The simulation of the shielded protocol does not attempt to model any
actual privacy properties.
"""
from collections import deque
from __future__ import annotations
from typing import Iterable, Optional
from collections.abc import Sequence
from dataclasses import dataclass
from enum import Enum, auto
from itertools import chain
from collections import deque
from itertools import chain, islice
from sys import version_info
from ..util import Unique
@ -34,7 +39,7 @@ class BCTransaction:
@dataclass(frozen=True)
class _TXO:
tx: 'BCTransaction'
tx: BCTransaction
index: int
value: int
@ -51,8 +56,14 @@ class BCTransaction:
"""
value: int
def __init__(self, transparent_inputs, transparent_output_values, shielded_inputs,
shielded_output_values, fee, anchor=None, issuance=0):
def __init__(self,
transparent_inputs: Sequence[BCTransaction._TXO],
transparent_output_values: Sequence[int],
shielded_inputs: Sequence[BCTransaction._Note],
shielded_output_values: Sequence[int],
fee: int,
anchor: Optional[BCContext]=None,
issuance: int=0):
"""
Constructs a `BCTransaction` with the given transparent inputs, transparent
output values, anchor, shielded inputs, shielded output values, fee, and
@ -78,8 +89,6 @@ class BCTransaction:
assert fee >= 0 or coinbase
assert issuance == 0 or coinbase
assert all((v >= 0 for v in chain(transparent_output_values, shielded_output_values)))
assert all((isinstance(txin, self._TXO) for txin in transparent_inputs))
assert all((isinstance(note, self._Note) for note in shielded_inputs))
assert (
sum((txin.value for txin in transparent_inputs))
+ sum((note.value for note in shielded_inputs))
@ -88,7 +97,8 @@ class BCTransaction:
+ sum(shielded_output_values)
+ fee
)
assert anchor is None if len(shielded_inputs) == 0 else anchor.can_spend(shielded_inputs)
assert anchor is None if len(shielded_inputs) == 0 else (
anchor is not None and anchor.can_spend(shielded_inputs))
self.transparent_inputs = transparent_inputs
self.transparent_outputs = [self._TXO(self, i, v)
@ -99,23 +109,23 @@ class BCTransaction:
self.anchor = anchor
self.issuance = issuance
def transparent_input(self, index):
def transparent_input(self, index: int) -> BCTransaction._TXO:
"""Returns the transparent input TXO with the given index."""
return self.transparent_inputs[index]
def transparent_output(self, index):
def transparent_output(self, index: int) -> BCTransaction._TXO:
"""Returns the transparent output TXO with the given index."""
return self.transparent_outputs[index]
def shielded_input(self, index):
def shielded_input(self, index: int) -> BCTransaction._Note:
"""Returns the shielded input note with the given index."""
return self.shielded_inputs[index]
def shielded_output(self, index):
def shielded_output(self, index: int) -> BCTransaction._Note:
"""Returns the shielded output note with the given index."""
return self.shielded_outputs[index]
def is_coinbase(self):
def is_coinbase(self) -> bool:
"""
Returns `True` if this is a coinbase transaction (it has no inputs).
"""
@ -140,27 +150,27 @@ class BCContext:
def __init__(self):
"""Constructs an empty `BCContext`."""
self.transactions = deque() # of BCTransaction
self.utxo_set = set() # of BCTransaction._TXO
self.transactions: deque[BCTransaction] = deque()
self.utxo_set: set[BCTransaction._TXO] = set()
# Since dicts are insertion-ordered, this models the sequence in which
# notes are committed as well as their spentness.
self.notes = {} # Note -> Spent | Unspent
self.notes: dict[BCTransaction._Note, Spentness] = {}
self.total_issuance = 0
def committed_notes(self):
def committed_notes(self) -> list[(BCTransaction._Note, Spentness)]:
"""
Returns a list of (`Note`, `Spentness`) for notes added to this context,
preserving the commitment order.
"""
return list(self.notes.items())
def can_spend(self, tospend):
def can_spend(self, tospend: Iterable[BCTransaction._Note]) -> bool:
"""Can all of the notes in `tospend` be spent in this context?"""
return all((self.notes.get(note) == Spentness.Unspent for note in tospend))
def _check(self, tx):
def _check(self, tx: BCTransaction) -> tuple[bool, set[BCTransaction._TXO]]:
"""
Checks whether `tx` is valid. To avoid recomputation, this returns
a pair of the validity, and the set of transparent inputs of `tx`.
@ -169,11 +179,11 @@ class BCContext:
valid = txins.issubset(self.utxo_set) and self.can_spend(tx.shielded_inputs)
return (valid, txins)
def is_valid(self, tx):
def is_valid(self, tx: BCTransaction) -> bool:
"""Is `tx` valid in this context?"""
return self._check(tx)[0]
def add_if_valid(self, tx):
def add_if_valid(self, tx: BCTransaction) -> bool:
"""
If `tx` is valid in this context, add it to the context and return `True`.
Otherwise leave the context unchanged and return `False`.
@ -194,7 +204,7 @@ class BCContext:
return valid
def copy(self):
def copy(self) -> BCContext:
"""Returns an independent copy of this `BCContext`."""
ctx = BCContext()
ctx.transactions = self.transactions.copy()
@ -207,7 +217,11 @@ class BCContext:
class BCBlock:
"""A block in a best-chain protocol."""
def __init__(self, parent, added_score, transactions, allow_invalid=False):
def __init__(self,
parent: Optional[BCBlock],
added_score: int,
transactions: Sequence[BCTransaction],
allow_invalid: bool=False):
"""
Constructs a `BCBlock` with the given parent block, score relative to the
parent, and sequence of transactions. `transactions` must not be modified
@ -215,22 +229,23 @@ class BCBlock:
If `allow_invalid` is set, the block need not be valid.
Use `parent=None` to construct the genesis block.
"""
assert all((isinstance(tx, BCTransaction) for tx in transactions))
self.parent = parent
self.score = (0 if parent is None else self.parent.score) + added_score
self.score = added_score
if self.parent is not None:
self.score += self.parent.score
self.transactions = transactions
self.hash = BlockHash()
if not allow_invalid:
self.assert_noncontextually_valid()
def assert_noncontextually_valid(self):
def assert_noncontextually_valid(self) -> None:
"""Assert that non-contextual consensus rules are satisfied for this block."""
assert len(self.transactions) > 0
assert self.transactions[0].is_coinbase()
assert not any((tx.is_coinbase() for tx in self.transactions[1:]))
assert not any((tx.is_coinbase() for tx in islice(self.transactions, 1, None)))
assert sum((tx.fee for tx in self.transactions)) == 0
def is_noncontextually_valid(self):
def is_noncontextually_valid(self) -> bool:
"""Are non-contextual consensus rules satisfied for this block?"""
try:
self.assert_noncontextually_valid()
@ -260,7 +275,7 @@ import unittest
class TestBC(unittest.TestCase):
def test_basic(self):
def test_basic(self) -> None:
ctx = BCContext()
coinbase_tx0 = BCTransaction([], [10], [], [], 0, issuance=10)
self.assertTrue(ctx.add_if_valid(coinbase_tx0))

9
simtfl/bc/demo.py
View File

@ -1,7 +1,14 @@
"""
This demo just runs the `simtfl.bc` unit tests for now.
"""
from __future__ import annotations
import unittest
def run():
def run() -> None:
"""
Runs the demo.
"""

174
simtfl/bft/__init__.py Normal file
View File

@ -0,0 +1,174 @@
"""
Abstractions for Byzantine Fault-Tolerant protocols.
The model of a BFT protocol assumed here was written for the purpose
of Crosslink, based on the example of adapted-Streamlet ([CS2020] as
modified in [Crosslink]). It might not be sufficient for other BFT
protocols — but that's okay; it's a prototype.
[CS2020] https://eprint.iacr.org/2020/088.pdf
[Crosslink] https://hackmd.io/JqENg--qSmyqRt_RqY7Whw?view
"""
from __future__ import annotations
def two_thirds_threshold(n: int) -> int:
"""
Calculate the notarization threshold used in most permissioned BFT protocols:
`ceiling(n * 2/3)`.
"""
return (n * 2 + 2) // 3
class PermissionedBFTBase:
"""
This class is used for the genesis block in a permissioned BFT protocol
(which is taken to be notarized, and therefore valid, by definition).
It is also used as a base class for other BFT block and proposal classes.
"""
def __init__(self, n: int, t: int):
"""
Constructs a genesis block for a permissioned BFT protocol with
`n` nodes, of which at least `t` must sign each proposal.
"""
self.n = n
self.t = t
self.parent = None
def last_final(self) -> PermissionedBFTBase:
"""
Returns the last final block in this block's ancestor chain.
For the genesis block, this is itself.
"""
return self
class PermissionedBFTBlock(PermissionedBFTBase):
"""
A block for a BFT protocol. Each non-genesis block is based on a
notarized proposal, and in practice consists of the proposer's signature
over the notarized proposal.
Honest proposers must only ever sign at most one valid proposal for the
given epoch in which they are a proposer.
BFT blocks are taken to be notarized, and therefore valid, by definition.
"""
def __init__(self, proposal: PermissionedBFTProposal):
"""Constructs a `PermissionedBFTBlock` for the given proposal."""
super().__init__(proposal.n, proposal.t)
proposal.assert_notarized()
self.proposal = proposal
self.parent = proposal.parent
def last_final(self):
"""
Returns the last final block in this block's ancestor chain.
This should be overridden by subclasses; the default implementation
will (inefficiently) just return the genesis block.
"""
return self if self.parent is None else self.parent.last_final()
class PermissionedBFTProposal(PermissionedBFTBase):
"""A proposal for a BFT protocol."""
def __init__(self, parent: PermissionedBFTBase):
"""
Constructs a `PermissionedBFTProposal` with the given parent
`PermissionedBFTBlock`. The parameters are determined by the parent
block.
"""
super().__init__(parent.n, parent.t)
self.parent = parent
self.signers = set()
def assert_valid(self) -> None:
"""
Assert that this proposal is valid. This does not assert that it is
notarized. This should be overridden by subclasses.
"""
pass
def is_valid(self) -> bool:
"""Is this proposal valid?"""
try:
self.assert_valid()
return True
except AssertionError:
return False
def assert_notarized(self) -> None:
"""
Assert that this proposal is notarized. A `PermissionedBFTProposal`
is notarized iff it is valid and has at least the threshold number of
signatures.
"""
self.assert_valid()
assert len(self.signers) >= self.t
def is_notarized(self) -> bool:
"""Is this proposal notarized?"""
try:
self.assert_notarized()
return True
except AssertionError:
return False
def add_signature(self, index: int) -> None:
"""
Record that the node with the given `index` has signed this proposal.
If the same node signs more than once, the subsequent signatures are
ignored.
"""
self.signers.add(index)
assert len(self.signers) <= self.n
__all__ = ['two_thirds_threshold', 'PermissionedBFTBase', 'PermissionedBFTBlock', 'PermissionedBFTProposal']
import unittest
class TestPermissionedBFT(unittest.TestCase):
def test_basic(self) -> None:
# Construct the genesis block.
genesis = PermissionedBFTBase(5, 2)
current = genesis
self.assertEqual(current.last_final(), genesis)
for _ in range(2):
proposal = PermissionedBFTProposal(current)
proposal.assert_valid()
self.assertTrue(proposal.is_valid())
self.assertFalse(proposal.is_notarized())
# not enough signatures
proposal.add_signature(0)
self.assertFalse(proposal.is_notarized())
# same index, so we still only have one signature
proposal.add_signature(0)
self.assertFalse(proposal.is_notarized())
# different index, now we have two signatures as required
proposal.add_signature(1)
proposal.assert_notarized()
self.assertTrue(proposal.is_notarized())
current = PermissionedBFTBlock(proposal)
self.assertEqual(current.last_final(), genesis)
def test_assertions(self) -> None:
genesis = PermissionedBFTBase(5, 2)
proposal = PermissionedBFTProposal(genesis)
self.assertRaises(AssertionError, PermissionedBFTBlock, proposal)
proposal.add_signature(0)
self.assertRaises(AssertionError, PermissionedBFTBlock, proposal)
proposal.add_signature(1)
_ = PermissionedBFTBlock(proposal)

180
simtfl/bft/streamlet/__init__.py Normal file
View File

@ -0,0 +1,180 @@
"""
An implementation of adapted-Streamlet ([CS2020] as modified in [Crosslink]).
[CS2020] https://eprint.iacr.org/2020/088.pdf
[Crosslink] https://hackmd.io/JqENg--qSmyqRt_RqY7Whw?view
"""
from __future__ import annotations
from typing import Optional
from collections.abc import Sequence
from .. import PermissionedBFTBase, PermissionedBFTBlock, PermissionedBFTProposal, \
two_thirds_threshold
class StreamletProposal(PermissionedBFTProposal):
"""An adapted-Streamlet proposal."""
def __init__(self, parent: StreamletBlock | StreamletGenesis, epoch: int):
"""
Constructs a `StreamletProposal` with the given parent `StreamletBlock`,
for the given `epoch`. The parameters are determined by the parent block.
A proposal must be for an epoch after its parent's epoch.
"""
super().__init__(parent)
assert epoch > parent.epoch
self.epoch = epoch
"""The epoch of this proposal."""
def __repr__(self) -> str:
return "StreamletProposal(parent=%r, epoch=%r)" % (self.parent, self.epoch)
class StreamletGenesis(PermissionedBFTBase):
"""An adapted-Streamlet genesis block."""
def __init__(self, n: int):
"""
Constructs a genesis block for adapted-Streamlet with `n` nodes.
"""
super().__init__(n, two_thirds_threshold(n))
self.epoch = 0
"""The genesis block has epoch 0."""
def __repr__(self) -> str:
return "StreamletGenesis(n=%r)" % (self.n,)
class StreamletBlock(PermissionedBFTBlock):
"""
An adapted-Streamlet block. Each non-genesis Streamlet block is
based on a notarized `StreamletProposal`.
`StreamletBlock`s are taken to be notarized by definition.
All validity conditions are enforced in the contructor.
"""
def __init__(self, proposal: StreamletProposal):
"""Constructs a `StreamletBlock` for the given proposal."""
super().__init__(proposal)
self.epoch = proposal.epoch
def last_final(self) -> StreamletBlock | StreamletGenesis:
"""
Returns the last final block in this block's ancestor chain.
In Streamlet this is the middle block of the last group of three
that were proposed in consecutive epochs.
"""
last = self
if last.parent is None:
return last
middle = last.parent
if middle.parent is None:
return middle
first = middle.parent
while True:
if first.parent is None:
return first
if (first.epoch + 1, middle.epoch + 1) == (middle.epoch, last.epoch):
return middle
(first, middle, last) = (first.parent, first, middle)
def __repr__(self) -> str:
return "StreamletBlock(proposal=%r)" % (self.proposal,)
import unittest
from itertools import count
class TestStreamlet(unittest.TestCase):
def test_simple(self) -> None:
"""
Very simple example.
0 --- 1 --- 2 --- 3
"""
self._test_last_final([0, 1, 2], [0, 0, 2])
def test_figure_1(self) -> None:
"""
Figure 1: Streamlet finalization example (without the invalid 'X' proposal).
0 --- 2 --- 5 --- 6 --- 7
\
-- 1 --- 3
0 - Genesis
N - Notarized block
This diagram implies the epoch 6 block is the last-final block in the
context of the epoch 7 block, because it is in the middle of 3 blocks
with consecutive epoch numbers, and 6 is the most recent such block.
(We don't include the block/proposal with the red X because that's not
what we're testing.)
"""
self._test_last_final([0, 0, 1, None, 2, 5, 6], [0, 0, 0, 0, 0, 0, 6])
def test_complex(self) -> None:
"""
Safety Violation: due to three simultaneous properties:
- 6 is `last_final` in the context of 7
- 9 is `last_final` in the context of 10
- 9 is not a descendant of 6
0 --- 2 --- 5 --- 6 --- 7
\
-- 1 --- 3 --- 8 --- 9 --- 10
"""
self._test_last_final([0, 0, 1, None, 2, 5, 6, 3, 8, 9], [0, 0, 0, 0, 0, 0, 6, 0, 0, 9])
def _test_last_final(self, parent_map: Sequence[Optional[int]], final_map: Sequence[int]) -> None:
"""
This test constructs a tree of proposals with structure determined by
`parent_map`, and asserts `block.last_final()` matches the structure
determined by `final_map`.
parent_map: sequence of parent epoch numbers
final_map: sequence of final epoch numbers
"""
assert len(parent_map) == len(final_map)
# Construct the genesis block.
genesis = StreamletGenesis(3)
current = genesis
self.assertEqual(current.last_final(), genesis)
blocks = [genesis]
for (epoch, parent_epoch, final_epoch) in zip(count(1), parent_map, final_map):
if parent_epoch is None:
blocks.append(None)
continue
parent = blocks[parent_epoch]
assert parent is not None
proposal = StreamletProposal(parent, epoch)
proposal.assert_valid()
self.assertTrue(proposal.is_valid())
self.assertFalse(proposal.is_notarized())
# not enough signatures
proposal.add_signature(0)
self.assertFalse(proposal.is_notarized())
# same index, so we still only have one signature
proposal.add_signature(0)
self.assertFalse(proposal.is_notarized())
# different index, now we have two signatures as required
proposal.add_signature(1)
proposal.assert_notarized()
self.assertTrue(proposal.is_notarized())
current = StreamletBlock(proposal)
blocks.append(current)
self.assertEqual(current.last_final(), blocks[final_epoch])

73
simtfl/bft/streamlet/node.py Normal file
View File

@ -0,0 +1,73 @@
"""
An adapted-Streamlet node.
"""
from __future__ import annotations
from ...node import SequentialNode
from ...message import Message, PayloadMessage
from ...util import skip, ProcessEffect
from . import StreamletGenesis, StreamletBlock, StreamletProposal
class Echo(PayloadMessage):
"""
An echo of another message. Streamlet requires nodes to broadcast each received
non-echo message to every other node.
"""
pass
class StreamletNode(SequentialNode):
"""
A Streamlet node.
"""
def __init__(self, genesis: StreamletGenesis):
"""
Constructs a Streamlet node with parameters taken from the given `genesis`
block (an instance of `StreamletGenesis`).
"""
assert genesis.epoch == 0
self.genesis = genesis
self.voted_epoch = genesis.epoch
def handle(self, sender: int, message: Message) -> ProcessEffect:
"""
(process) Message handler for a Streamlet node:
* `Echo` messages are unwrapped and treated like the original message.
* Non-`Echo` messages are implicitly echoed to every other node.
(This causes the number of messages to blow up by a factor of `n`,
but it's what the Streamlet paper specifies and is necessary for
its liveness proof.)
* Received non-duplicate proposals may cause us to send a `Vote`.
* ...
"""
if isinstance(message, Echo):
message = message.payload
else:
yield from self.broadcast(Echo(message))
if isinstance(message, StreamletProposal):
yield from self.handle_proposal(message)
elif isinstance(message, StreamletBlock):
yield from self.handle_block(message)
else:
yield from super().handle(sender, message)
def handle_proposal(self, proposal: StreamletProposal) -> ProcessEffect:
"""
(process) If we already voted in the epoch specified by the proposal or a
later epoch, ignore this proposal.
"""
if proposal.epoch <= self.voted_epoch:
self.log("handle",
f"received proposal for epoch {proposal.epoch} but we already voted in epoch {self.voted_epoch}")
return skip()
return skip()
def handle_block(self, block: StreamletBlock) -> ProcessEffect:
raise NotImplementedError

17
simtfl/demo.py
View File

@ -2,11 +2,16 @@
A simple demo of message passing.
"""
from __future__ import annotations
from simpy import Environment
from .message import PayloadMessage
from .logging import PrintLogger
from .message import Message, PayloadMessage
from .network import Network
from .node import PassiveNode, SequentialNode
from .util import ProcessEffect
class Ping(PayloadMessage):
@ -27,7 +32,7 @@ class PingNode(PassiveNode):
"""
A node that sends pings.
"""
def run(self):
def run(self) -> ProcessEffect:
"""
(process) Sends two Ping messages to every node.
"""
@ -42,7 +47,7 @@ class PongNode(SequentialNode):
"""
A node that responds to pings sequentially.
"""
def handle(self, sender, message):
def handle(self, sender: int, message: Message) -> ProcessEffect:
"""
(process) Handles a Ping message by sending back a Pong message with the
same payload.
@ -54,12 +59,12 @@ class PongNode(SequentialNode):
yield from super().handle(sender, message)
def run():
def run() -> None:
"""
Runs the demo.
"""
network = Network(Environment(), delay=4)
for i in range(10):
network = Network(Environment(), delay=4, logger=PrintLogger())
for _ in range(10):
network.add_node(PongNode())
network.add_node(PingNode())

46
simtfl/logging.py Normal file
View File

@ -0,0 +1,46 @@
"""
Utility classes for logging.
"""
from __future__ import annotations
from typing import Optional, TextIO
from numbers import Number
import sys
class Logger:
"""
A logger that does nothing. This class can be used directly or as a base
for other logger classes.
"""
def header(self) -> None:
"""Do not print a header."""
pass
def log(self, now: Number, ident: int, event: str, detail: str) -> None:
"""Do not log."""
pass
class PrintLogger(Logger):
"""A logger that prints to a stream."""
def __init__(self, out: Optional[TextIO]=None):
"""
Constructs a `PrintLogger` that prints to `out` (by default `sys.stdout`).
"""
if out is None:
out = sys.stdout
self.out = out
def header(self) -> None:
"""Print a table header."""
print()
print(" Time | Node | Event | Detail", file=self.out)
def log(self, now: Number, ident: int, event: str, detail: str) -> None:
"""Print a log line."""
print(f"{now:5d} | {ident:4d} | {event:10} | {detail}", file=self.out)

13
simtfl/message.py
View File

@ -2,12 +2,21 @@
Base classes for messages.
"""
from dataclasses import dataclass
from __future__ import annotations
from typing import Any
from dataclasses import dataclass
class Message:
"""
Base class for messages.
"""
pass
@dataclass(frozen=True)
class PayloadMessage:
class PayloadMessage(Message):
"""
A message with an arbitrary payload.
"""

138
simtfl/network.py
View File

@ -2,35 +2,115 @@
Framework for message passing in a network of nodes.
"""
from .util import skip
from __future__ import annotations
from typing import Optional
from numbers import Number
from simpy import Environment
from simpy.events import Timeout, Process
from .message import Message
from .util import skip, ProcessEffect
from .logging import Logger
class Node:
"""
A base class for nodes. This class is intended to be subclassed.
"""
def initialize(self, ident: int, env: Environment, network: Network):
"""
Initializes a `Node` with the given ident, `simpy.Environment`, and `Network`.
Nodes are initialized when they are added to a `Network`. The implementation
in this class sets the `ident`, `env`, and `network` fields.
"""
self.ident = ident
self.env = env
self.network = network
def __str__(self):
return f"{self.__class__.__name__}"
def log(self, event: str, detail: str):
"""
Logs an event described by `event` and `detail` for this node.
"""
self.network.log(self.ident, event, detail)
def send(self, target: int, message: Message, delay: Optional[Number]=None) -> ProcessEffect:
"""
(process) This method can be overridden to intercept messages being sent
by this node. The implementation in this class calls `self.network.send`
with this node as the sender.
"""
return self.network.send(self.ident, target, message, delay=delay)
def broadcast(self, message: Message, delay: Optional[Number]=None) -> ProcessEffect:
"""
(process) This method can be overridden to intercept messages being broadcast
by this node. The implementation in this class calls `self.network.broadcast`
with this node as the sender.
"""
return self.network.broadcast(self.ident, message, delay=delay)
def receive(self, sender: int, message: Message) -> ProcessEffect:
"""
(process) This method can be overridden to intercept messages being received
by this node. The implementation in this class calls `self.handle`.
"""
return self.handle(sender, message)
def handle(self, sender: int, message: Message) -> ProcessEffect:
"""
(process) Handles a message. Subclasses must implement this method.
"""
raise NotImplementedError
def run(self) -> ProcessEffect:
"""
(process) Runs the node. Subclasses must implement this method.
"""
raise NotImplementedError
class Network:
"""
Simulate the network layer.
"""
def __init__(self, env, nodes=None, delay=1):
def __init__(self, env: Environment, nodes: Optional[list[Node]]=None, delay: Number=1,
logger: Logger=Logger()):
"""
Constructs a Network with the given `simpy.Environment`, and optionally
a set of initial nodes and a message propagation delay.
Constructs a `Network` with the given `simpy.Environment`, and optionally
a set of initial nodes, message propagation delay, and logger.
"""
self.env = env
self.nodes = nodes or []
self.delay = delay
self._logger = logger
logger.header()
def num_nodes(self):
def log(self, ident: int, event: str, detail: str) -> None:
"""
Logs an event described by `event` and `detail` for the node with the
given `ident`.
"""
self._logger.log(self.env.now, ident, event, detail)
def num_nodes(self) -> int:
"""
Returns the number of nodes.
"""
return len(self.nodes)
def node(self, ident):
def node(self, ident: int) -> Node:
"""
Returns the node with the given integer ident.
"""
return self.nodes[ident]
def add_node(self, node):
def add_node(self, node: Node) -> None:
"""
Adds a node with the next available ident.
"""
@ -38,31 +118,30 @@ class Network:
self.nodes.append(node)
node.initialize(ident, self.env, self)
def _start(self, node):
def _start(self, node: Node) -> None:
"""
Starts a process for the given node (which is assumed to
have already been added to this `Network`).
"""
print(f"T{self.env.now:5d}: starting {node.ident:2d}: {node}")
self.env.process(node.run())
self.log(node.ident, "start", str(node))
Process(self.env, node.run())
def start_node(self, ident):
def start_node(self, ident: int) -> None:
"""
Starts a process for the node with the given ident.
A given node should only be started once.
"""
self._start(self.nodes[ident])
def start_all_nodes(self):
def start_all_nodes(self) -> None:
"""
Starts a process for each node.
A given node should only be started once.
"""
print()
for node in self.nodes:
self._start(node)
def run_all(self, *args, **kwargs):
def run_all(self, *args, **kwargs) -> None:
"""
Convenience method to start a process for each node, then start
the simulation. Takes the same arguments as `simpy.Environment.run`.
@ -70,7 +149,7 @@ class Network:
self.start_all_nodes()
self.env.run(*args, **kwargs)
def send(self, sender, target, message, delay=None):
def send(self, sender: int, target: int, message: Message, delay: Optional[Number]=None) -> ProcessEffect:
"""
(process) Sends a message to the node with ident `target`, from the node
with ident `sender`. The message propagation delay is normally given by
@ -78,16 +157,35 @@ class Network:
"""
if delay is None:
delay = self.delay
print(f"T{self.env.now:5d}: sending {sender:2d} -> {target:2d} delay {delay:2d}: {message}")
self.log(sender, "send", f"to {target:2d} with delay {delay:2d}: {message}")
# Run `convey` in a new process without waiting.
self.env.process(self.convey(delay, sender, target, message))
Process(self.env, self.convey(delay, sender, target, message))
# Sending is currently instantaneous.
# TODO: make it take some time on the sending node.
return skip()
def convey(self, delay, sender, target, message):
def broadcast(self, sender: int, message: Message, delay: Optional[Number]=None) -> ProcessEffect:
"""
(process) Broadcasts a message to every other node. The message
propagation delay is normally given by `self.delay`, but can be
overridden by the `delay` parameter.
"""
if delay is None:
delay = self.delay
self.log(sender, "broadcast", f"to * with delay {delay:2d}: {message}")
# Run `convey` in a new process for each node.
for target in range(self.num_nodes()):
if target != sender:
Process(self.env, self.convey(delay, sender, target, message))
# Broadcasting is currently instantaneous.
# TODO: make it take some time on the sending node.
return skip()
def convey(self, delay: Number, sender: int, target: int, message: Message) -> ProcessEffect:
"""
(process) Conveys a message to the node with ident `target`, from the node
with ident `sender`, after waiting for the given message propagation delay.
@ -95,6 +193,6 @@ class Network:
after the message has been handled by the target node. The caller should
not depend on when it completes.
"""
yield self.env.timeout(delay)
print(f"T{self.env.now:5d}: receiving {sender:2d} -> {target:2d} delay {delay:2d}: {message}")
yield Timeout(self.env, delay)
self.log(target, "receive", f"from {sender:2d} with delay {delay:2d}: {message}")
yield from self.nodes[target].receive(sender, message)

115
simtfl/node.py
View File

@ -2,12 +2,19 @@
Base classes for node implementations.
"""
from collections import deque
from .util import skip
from __future__ import annotations
from numbers import Number
from simpy import Environment
from simpy.events import Event
from .message import Message
from .network import Network, Node
from .util import skip, ProcessEffect
class PassiveNode:
class PassiveNode(Node):
"""
A node that processes messages concurrently. By default it sends no
messages and does nothing with received messages. This class is
@ -20,33 +27,8 @@ class PassiveNode:
Note that the simulation is deterministic regardless of which option
is selected.
"""
def initialize(self, ident, env, network):
"""
Initializes a `PassiveNode` with the given ident, `simpy.Environment`,
and `Network`. Nodes are initialized when they are added to a `Network`.
"""
self.ident = ident
self.env = env
self.network = network
def __str__(self):
return f"{self.__class__.__name__}"
def send(self, target, message, delay=None):
"""
(process) This method can be overridden to intercept messages being sent
by this node. The implementation in this class calls `self.network.send`.
"""
return self.network.send(self.ident, target, message, delay=delay)
def receive(self, sender, message):
"""
(process) This method can be overridden to intercept messages being received
by this node. The implementation in this class calls `self.handle`.
"""
return self.handle(sender, message)
def handle(self, sender, message):
def handle(self, sender: int, message: Message) -> ProcessEffect:
"""
(process) Handles a message by doing nothing. Note that the handling of
each message, and the `run` method, are in separate simpy processes. That
@ -55,28 +37,30 @@ class PassiveNode:
"""
return skip()
def run(self):
def run(self) -> ProcessEffect:
"""
(process) Runs by doing nothing.
"""
return skip()
class SequentialNode(PassiveNode):
class SequentialNode(Node):
"""
A node that processes messages sequentially. By default it sends no
messages and does nothing with received messages. This class is
intended to be subclassed.
"""
def initialize(self, ident, env, network):
def initialize(self, ident: int, env: Environment, network: Network):
"""
Initializes a `SequentialNode` with the given `simpy.Environment` and `Network`.
Initializes a `SequentialNode` with the given ident, `simpy.Environment`,
and `Network`. Nodes are initialized when they are added to a `Network`.
"""
super().initialize(ident, env, network)
self._mailbox = deque()
self._wakeup = env.event()
self._mailbox: deque[tuple[int, Message]] = deque()
self._wakeup = Event(self.env)
def receive(self, sender, message):
def receive(self, sender: int, message: Message) -> ProcessEffect:
"""
(process) Add incoming messages to the mailbox.
"""
@ -87,17 +71,15 @@ class SequentialNode(PassiveNode):
pass
return skip()
def handle(self, sender, message):
def handle(self, sender: int, message: Message) -> ProcessEffect:
"""
(process) Handles a message by doing nothing. Messages are handled
sequentially; that is, handling of the next message will be blocked
on this process.
"""
# This is the same implementation as `PassiveNode`, but the documentation
# is different.
return skip()
def run(self):
def run(self) -> ProcessEffect:
"""
(process) Repeatedly handle incoming messages.
If a subclass needs to perform tasks in parallel with message handling,
@ -107,84 +89,92 @@ class SequentialNode(PassiveNode):
while True:
while len(self._mailbox) > 0:
(sender, message) = self._mailbox.popleft()
print(f"T{self.env.now:5d}: handling {sender:2d} -> {self.ident:2d}: {message}")
self.log("handle", f"from {sender:2d}: {message}")
yield from self.handle(sender, message)
# This naive implementation is fine because we have no actual
# concurrency.
self._wakeup = self.env.event()
self._wakeup = Event(self.env)
yield self._wakeup
__all__ = ['PassiveNode', 'SequentialNode']
from simpy import Environment
import unittest
from collections import deque
from simpy.events import Timeout
from .logging import PrintLogger
from .message import PayloadMessage
from .network import Network
class PassiveReceiverTestNode(PassiveNode):
def __init__(self):
super().__init__()
self.handled = deque()
self.handled: deque[tuple[int, Message, Number]] = deque()
def handle(self, sender, message):
def handle(self, sender: int, message: Message) -> ProcessEffect:
# Record when each message is handled.
self.handled.append((sender, message, self.env.now))
# The handler takes 3 time units.
yield self.env.timeout(3)
yield Timeout(self.env, 3)
class SequentialReceiverTestNode(SequentialNode):
def __init__(self):
super().__init__()
self.handled = deque()
self.handled: deque[tuple[int, Message, Number]] = deque()
def handle(self, sender, message):
def handle(self, sender: int, message: Message) -> ProcessEffect:
# Record when each message is handled.
self.handled.append((sender, message, self.env.now))
# The handler takes 3 time units.
yield self.env.timeout(3)
yield Timeout(self.env, 3)
class SenderTestNode(PassiveNode):
def run(self):
def run(self) -> ProcessEffect:
# We send messages at times 0, 1, 2. Since the message
# propagation delay is 1 (the default), they will be
# received at times 1, 2, 3.
for i in range(3):
yield from self.send(0, PayloadMessage(i))
yield self.env.timeout(1)
yield Timeout(self.env, 1)
# Test overriding the propagation delay. This message
# is sent at time 3 and received at time 11.
yield from self.send(0, PayloadMessage(3), delay=8)
# is sent at time 3 and received at time 14.
yield from self.send(0, PayloadMessage(3), delay=11)
yield Timeout(self.env, 1)
# This message is broadcast at time 4 and received at time 5.
yield from self.broadcast(PayloadMessage(4))
class TestFramework(unittest.TestCase):
def _test_node(self, receiver_node, expected):
network = Network(Environment())
def _test_node(self,
receiver_node: Node,
expected: list[tuple[int, Message, Number]]) -> None:
network = Network(Environment(), logger=PrintLogger())
network.add_node(receiver_node)
network.add_node(SenderTestNode())
network.run_all()
self.assertEqual(list(network.node(0).handled), expected)
def test_passive_node(self):
# A PassiveNode subclass does not block on handling of
def test_passive_node(self) -> None:
# A `PassiveNode` subclass does not block on handling of
# previous messages, so it handles each message immediately
# when it is received.
self._test_node(PassiveReceiverTestNode(), [
(1, PayloadMessage(0), 1),
(1, PayloadMessage(1), 2),
(1, PayloadMessage(2), 3),
(1, PayloadMessage(3), 11),
(1, PayloadMessage(4), 5),
(1, PayloadMessage(3), 14),
])
def test_sequential_node(self):
# A SequentialNode subclass *does* block on handling of
def test_sequential_node(self) -> None:
# A `SequentialNode` subclass *does* block on handling of
# previous messages. It handles the messages as soon as
# possible after they are received subject to that blocking,
# so they will be handled at intervals of 3 time units.
@ -192,5 +182,6 @@ class TestFramework(unittest.TestCase):
(1, PayloadMessage(0), 1),
(1, PayloadMessage(1), 4),
(1, PayloadMessage(2), 7),
(1, PayloadMessage(3), 11),
(1, PayloadMessage(4), 10),
(1, PayloadMessage(3), 14),
])

14
simtfl/util.py
View File

@ -3,7 +3,15 @@ Utilities.
"""
def skip():
from __future__ import annotations
from typing import Generator, TypeAlias
from simpy import Event
ProcessEffect: TypeAlias = Generator[Event, None, None]
def skip() -> ProcessEffect:
"""
(process) Does nothing.
"""
@ -18,8 +26,8 @@ class Unique:
Instances of this class are hashable. When subclassing as a dataclass, use
`@dataclass(eq=False)` to preserve hashability.
"""
def __eq__(self, other):
def __eq__(self, other: Unique):
return self == other
def __hash__(self):
def __hash__(self) -> int:
return id(self)