* liquidator: randomly select token/perps for rebalancing to avoid failing at every try if one token is having an issue (#921)
(cherry picked from commit e3a7ed9e32)
* liquidator: do not panic if token or perp rebalancing fails (#927)
(cherry picked from commit 0b7e62e671)
* Liquidator: add Sanctum swap (#919)
liquidator: add sanctum swap
(cherry picked from commit 01d5237162)
* liquidator: add more LST for sanctum swap (#944)
(cherry picked from commit c0b61b3b37)
* liquidator: forcefully exit process if snapshot job die
* client: return snapshot_job join handle so it can be watched for early unexpected exit
(cherry picked from commit 2520c7d095)
liquidator: split TCS triggering and liquidation job
Concurrent execution of candidate lookup and tx building/sending
- Also added an health assertion IX to protect liqor in multi liquidation scenario
- And a timeout for jupiter v6 queries (avoid blocking liquidation because of slow TCS)
* service-mango-health: add a new service that compute health for all mango accounts and persist an history in postgres sql
(cherry picked from commit 6aa8a883f4)
* add RaydiumCLMM oracle type
* add deserializing raydium pool state
* test: consolidate clmm oralce unit tests
* test: add failure check to test_orca_fallback_oracle
* test: add test_raydium_fallback_oracle
* restore get_quote_oracle()
* ensure RaydiumCLMM is a valid fallback type
rust client:
- add serum3 place order to command
- add serum3 create open orders command
- add serum3 close open orders command
- auto create serum3 open orders if needed when placing a new order
- auto close serum3 slot if needed when placing a new order & also close unused token if needed to place a serum3 order
- New permissionless instruction to regularly charge collateral fees
- Bank and group configuration to set rate and interval
- Keeper addition to call the instruction
* perp: auto close perp market account when needing to open a new one with no slot available
* rust_client: do not send health accounts when deactivating a perp position (not needed on program side)
* rust_client: add perp place order command
* rename usd_opt to usdc_opt in OracleAccountInfos
* use fallbacks when fetching bank+ price in AccountFetcher struct
* feat: add derive_fallback_oracle_keys to MangoGroupContext
* test: properly assert failure CU in test_health_compute_tokens_fallback_oracles
* provide fallback oracle accounts in the rust client
* liquidator: update for fallback oracles
* set fallback config in mango services
* support fallback oracles in rust settler + keeper
* fix send error related to fetching fallbacks dynamically in tcs_start
* drop derive_fallback_oracle_keys_sync
* add fetch_multiple_accounts to AccountFetcher trait
* revert client::new() api
* deriving oracle keys uses account_fetcher
* use client helpers for deriving health_check account_metas
* add health_cache helper to mango client
* add get_slot to account_fetcher
* lint
* cached account fetcher only fetches uncached accounts
* ensure keeper client does not use cached oracles for staleness checks
* address minor review comments
* create unique job keys for CachedAccountFetcher.fetch_multiple_accounts
* fmt
* improve hashing in CachedAccountFetcher.fetch_multiple_accounts
---------
Co-authored-by: Christian Kamm <mail@ckamm.de>
- keep track of different error types per key
- skip for a given duration once too many errors came in
- wipe if no new errors are seen for a while
- simplify logging of actively skipped cases
Serge Farny
Christian Kamm
riordanp
microwavedcola1
Lou-Kamades