💬 MarcoFalke commented on issue "make check errors on big endian OpenBSD 7.2":
(https://github.com/bitcoin/bitcoin/issues/26492#issuecomment-1543500303)
> Everything passed when running ./test/util/test_runner.py --verbose .
That means the issue originally reported is fixed and `make check` fully passes?
> ./test/functional/feature_addrman.py --tracerpc
Can you also try with `./test/functional/feature_addrman.py --tracerpc -l DEBUG` and then `./test/functional/combine_logs.py ./folder-tmp-test/foo/bar`?
(https://github.com/bitcoin/bitcoin/issues/26492#issuecomment-1543500303)
> Everything passed when running ./test/util/test_runner.py --verbose .
That means the issue originally reported is fixed and `make check` fully passes?
> ./test/functional/feature_addrman.py --tracerpc
Can you also try with `./test/functional/feature_addrman.py --tracerpc -l DEBUG` and then `./test/functional/combine_logs.py ./folder-tmp-test/foo/bar`?
⚠️ huzhenyuan opened an issue: "One core in CPU usage rate remains at 100% for a long time, causing serious delays in new blocks and forks"
(https://github.com/bitcoin/bitcoin/issues/27623)
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current behaviour
We are the technical team from AntPool. We found that when the Bitcoin wallet node processes the process message recently, the single-threaded CPU 100% causes block parsing delays, resulting in serious technical failures.
Our version number is the latest version 24.0.1。
Not only antpool, we also asked other mining pools and found this problem. This will lead to increasingly serious orpha
...
(https://github.com/bitcoin/bitcoin/issues/27623)
### Is there an existing issue for this?
- [X] I have searched the existing issues
### Current behaviour
We are the technical team from AntPool. We found that when the Bitcoin wallet node processes the process message recently, the single-threaded CPU 100% causes block parsing delays, resulting in serious technical failures.
Our version number is the latest version 24.0.1。
Not only antpool, we also asked other mining pools and found this problem. This will lead to increasingly serious orpha
...
💬 darosior commented on pull request "net processing: avoid serving non-announced txs as a result of a MEMPOOL message":
(https://github.com/bitcoin/bitcoin/pull/27602#issuecomment-1543552214)
I'm only a casual lurker on the net_processing side but i was under the impression we were thinking of the BIP35 as permissioned anyways. If we are already trusting the peer that sends use MEMPOOL message, why not just include everything we've got?
A practical counter-argument to this could be - yeah but at this time [around 20% to 30% of reachable nodes](https://github.com/bitcoin/bitcoin/pull/27426#issuecomment-1529678174) set `peerbloomfilters` so it's still worth closing an obvious fingerpr
...
(https://github.com/bitcoin/bitcoin/pull/27602#issuecomment-1543552214)
I'm only a casual lurker on the net_processing side but i was under the impression we were thinking of the BIP35 as permissioned anyways. If we are already trusting the peer that sends use MEMPOOL message, why not just include everything we've got?
A practical counter-argument to this could be - yeah but at this time [around 20% to 30% of reachable nodes](https://github.com/bitcoin/bitcoin/pull/27426#issuecomment-1529678174) set `peerbloomfilters` so it's still worth closing an obvious fingerpr
...
💬 willcl-ark commented on issue "One core in CPU usage rate remains at 100% for a long time, causing serious delays in new blocks and forks":
(https://github.com/bitcoin/bitcoin/issues/27623#issuecomment-1543579414)
Hello @huzhenyuan, thank you for the report.
There is a possible fix for similar symptoms currently open in https://github.com/bitcoin/bitcoin/pull/27610, but note that this is to alleviate high CPU usage originating from `SendMessages()`, not `ProcessMessages()` as you describe you have found in your case. However, both of these functions are run in the same thread `b-msghand` so they could be related or difficult to discern from one another.
If you are able and willing, you could try che
...
(https://github.com/bitcoin/bitcoin/issues/27623#issuecomment-1543579414)
Hello @huzhenyuan, thank you for the report.
There is a possible fix for similar symptoms currently open in https://github.com/bitcoin/bitcoin/pull/27610, but note that this is to alleviate high CPU usage originating from `SendMessages()`, not `ProcessMessages()` as you describe you have found in your case. However, both of these functions are run in the same thread `b-msghand` so they could be related or difficult to discern from one another.
If you are able and willing, you could try che
...
💬 FelixWeis commented on issue "One core in CPU usage rate remains at 100% for a long time, causing serious delays in new blocks and forks":
(https://github.com/bitcoin/bitcoin/issues/27623#issuecomment-1543649256)
thanks @willcl-ark, can you edit you comment and point to the relevant mitigation PR/issues.
(https://github.com/bitcoin/bitcoin/issues/27623#issuecomment-1543649256)
thanks @willcl-ark, can you edit you comment and point to the relevant mitigation PR/issues.
💬 fanquake commented on pull request "Fee estimation: avoid serving stale fee estimate":
(https://github.com/bitcoin/bitcoin/pull/27622#issuecomment-1543653619)
cc @instagibbs @TheBlueMatt
(https://github.com/bitcoin/bitcoin/pull/27622#issuecomment-1543653619)
cc @instagibbs @TheBlueMatt
🚀 fanquake merged a pull request: "refactor, kernel: Decouple ArgsManager from blockstorage"
(https://github.com/bitcoin/bitcoin/pull/27125)
(https://github.com/bitcoin/bitcoin/pull/27125)
💬 kouloumos commented on issue "rpc: Allow importing wallets by data instead of by filename":
(https://github.com/bitcoin/bitcoin/issues/27597#issuecomment-1543668223)
I might be missing something here, as I cannot understand how your latest question is related to your initial feature request.
It is common for backups to be in files, hence importing/restoring from a file. I understand the over-the-wire scenario, but if there is no strong use-case for such a feature, it makes it more difficult to gather interest.
Note that `importwallet` and the related `dumpwallet` RPC are only compatible with legacy wallets, and as there is a [proposed timeline for their
...
(https://github.com/bitcoin/bitcoin/issues/27597#issuecomment-1543668223)
I might be missing something here, as I cannot understand how your latest question is related to your initial feature request.
It is common for backups to be in files, hence importing/restoring from a file. I understand the over-the-wire scenario, but if there is no strong use-case for such a feature, it makes it more difficult to gather interest.
Note that `importwallet` and the related `dumpwallet` RPC are only compatible with legacy wallets, and as there is a [proposed timeline for their
...
💬 fanquake commented on pull request "kernel: Remove args, chainparams, chainparamsbase from kernel library":
(https://github.com/bitcoin/bitcoin/pull/27576#issuecomment-1543685914)
```bash
src/common/settings.h seems to be missing the expected include guard:
#ifndef BITCOIN_COMMON_SETTINGS_H
#define BITCOIN_COMMON_SETTINGS_H
...
#endif // BITCOIN_COMMON_SETTINGS_H
```
(https://github.com/bitcoin/bitcoin/pull/27576#issuecomment-1543685914)
```bash
src/common/settings.h seems to be missing the expected include guard:
#ifndef BITCOIN_COMMON_SETTINGS_H
#define BITCOIN_COMMON_SETTINGS_H
...
#endif // BITCOIN_COMMON_SETTINGS_H
```
💬 ajtowns commented on pull request "net processing: avoid serving non-announced txs as a result of a MEMPOOL message":
(https://github.com/bitcoin/bitcoin/pull/27602#discussion_r1190924614)
> But how about just recording the timestamp of the last MEMPOOL request ...
Patch might look something like...
<details><summary>Details</summary>
```diff
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -295,6 +295,8 @@ struct Peer {
* permitted if the peer has NetPermissionFlags::Mempool or we advertise
* NODE_BLOOM. See BIP35. */
bool m_send_mempool GUARDED_BY(m_tx_inventory_mutex){false};
+ std::chrono::microseconds m_last_memp
...
(https://github.com/bitcoin/bitcoin/pull/27602#discussion_r1190924614)
> But how about just recording the timestamp of the last MEMPOOL request ...
Patch might look something like...
<details><summary>Details</summary>
```diff
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -295,6 +295,8 @@ struct Peer {
* permitted if the peer has NetPermissionFlags::Mempool or we advertise
* NODE_BLOOM. See BIP35. */
bool m_send_mempool GUARDED_BY(m_tx_inventory_mutex){false};
+ std::chrono::microseconds m_last_memp
...
💬 MarcoFalke commented on issue "rpc: Allow importing wallets by data instead of by filename":
(https://github.com/bitcoin/bitcoin/issues/27597#issuecomment-1543700893)
Passing large data blobs, presumably multi-megabyte raw byte files, encoded as (presumably) hex, wrapped in json, over RPC is likely going to cause more issues than it fixes.
See also https://github.com/bitcoin/bitcoin/issues?q=%22Argument+list+too+long%22:
* GUI https://github.com/bitcoin/bitcoin/issues/17618
* Bash https://github.com/bitcoin/bitcoin/issues/4099
* Python https://github.com/bitcoin/bitcoin/issues/14767#issuecomment-619644381
(https://github.com/bitcoin/bitcoin/issues/27597#issuecomment-1543700893)
Passing large data blobs, presumably multi-megabyte raw byte files, encoded as (presumably) hex, wrapped in json, over RPC is likely going to cause more issues than it fixes.
See also https://github.com/bitcoin/bitcoin/issues?q=%22Argument+list+too+long%22:
* GUI https://github.com/bitcoin/bitcoin/issues/17618
* Bash https://github.com/bitcoin/bitcoin/issues/4099
* Python https://github.com/bitcoin/bitcoin/issues/14767#issuecomment-619644381
💬 jonatack commented on pull request "ci: Run iwyu on all src files":
(https://github.com/bitcoin/bitcoin/pull/27571#issuecomment-1543705947)
Concept ACK. I've been adding sources files to this list while working on each pull.
(https://github.com/bitcoin/bitcoin/pull/27571#issuecomment-1543705947)
Concept ACK. I've been adding sources files to this list while working on each pull.
📝 fanquake opened a pull request: "[23.2] Backports for rc1"
(https://github.com/bitcoin/bitcoin/pull/27624)
Collecting backports for rc1. Currently:
* https://github.com/bitcoin/bitcoin/pull/27608 (not a clean cherry-pick)
(https://github.com/bitcoin/bitcoin/pull/27624)
Collecting backports for rc1. Currently:
* https://github.com/bitcoin/bitcoin/pull/27608 (not a clean cherry-pick)
💬 jonatack commented on pull request "net, refactor: extract Network and BIP155Network logic to node/network":
(https://github.com/bitcoin/bitcoin/pull/27385#discussion_r1190956000)
Good point; removed that commit. Thanks for having a look @fanquake.
(https://github.com/bitcoin/bitcoin/pull/27385#discussion_r1190956000)
Good point; removed that commit. Thanks for having a look @fanquake.
💬 ajtowns commented on pull request "net processing: avoid serving non-announced txs as a result of a MEMPOOL message":
(https://github.com/bitcoin/bitcoin/pull/27602#discussion_r1190966758)
Okay, other idea: how about just making sure you only send at most 3500 mempool entries younger than 2 minutes, in response to a MEMPOOL message, and add them all to the filter, and don't worry if that causes overflow on results from a previous response?
(https://github.com/bitcoin/bitcoin/pull/27602#discussion_r1190966758)
Okay, other idea: how about just making sure you only send at most 3500 mempool entries younger than 2 minutes, in response to a MEMPOOL message, and add them all to the filter, and don't worry if that causes overflow on results from a previous response?
💬 willcl-ark commented on pull request "net processing: avoid serving non-announced txs as a result of a MEMPOOL message":
(https://github.com/bitcoin/bitcoin/pull/27602#issuecomment-1543749307)
> i was under the impression we were thinking of the BIP35 as permissioned anyways.
> ...
> For instance BTCPay/NBXplorer needs `whitelist=mempool` iirc.
Just on these points, currently the MEMPOOL message will be responded to _either_ if you have the whitelist permission `mempool` on the peer **or** if you have enabled NODE_BLOOM (with `-peerbloomfilters`). In the latter case I don't think we can accurately describe these as "trusted peers". See the changes in [`4581a68` (#27559)](https://
...
(https://github.com/bitcoin/bitcoin/pull/27602#issuecomment-1543749307)
> i was under the impression we were thinking of the BIP35 as permissioned anyways.
> ...
> For instance BTCPay/NBXplorer needs `whitelist=mempool` iirc.
Just on these points, currently the MEMPOOL message will be responded to _either_ if you have the whitelist permission `mempool` on the peer **or** if you have enabled NODE_BLOOM (with `-peerbloomfilters`). In the latter case I don't think we can accurately describe these as "trusted peers". See the changes in [`4581a68` (#27559)](https://
...
💬 MarcoFalke commented on pull request "net processing: avoid serving non-announced txs as a result of a MEMPOOL message":
(https://github.com/bitcoin/bitcoin/pull/27602#discussion_r1190981465)
> how about just making sure you only send at most 3500 mempool entries younger than 2 minutes, in response to a MEMPOOL message
Wouldn't that imply that peers are now expected to send the `mempool` message in a loop to detect if there are more transactions to get? If yes, I wonder if it conflicts with the idea to potentially only allow a single `mempool` message per connection.
(https://github.com/bitcoin/bitcoin/pull/27602#discussion_r1190981465)
> how about just making sure you only send at most 3500 mempool entries younger than 2 minutes, in response to a MEMPOOL message
Wouldn't that imply that peers are now expected to send the `mempool` message in a loop to detect if there are more transactions to get? If yes, I wonder if it conflicts with the idea to potentially only allow a single `mempool` message per connection.
💬 michaelfolkson commented on pull request "add ryanofsky to trusted-keys":
(https://github.com/bitcoin/bitcoin/pull/27604#issuecomment-1543765335)
Also for posterity: https://gist.github.com/michaelfolkson/81df14508ed3ad3232da133e154d6873
The maintainer role was always supposed to be janitorial one and was not elevated above the many long term contributors on the project. A hundred(s) billion dollar ecosystem shouldn't depend on the whims of a small group of maintainers who refuse to engage on decisions that not only impact the project but potentially the entire ecosystem (currently valued in the hundreds of billions of dollars). The fa
...
(https://github.com/bitcoin/bitcoin/pull/27604#issuecomment-1543765335)
Also for posterity: https://gist.github.com/michaelfolkson/81df14508ed3ad3232da133e154d6873
The maintainer role was always supposed to be janitorial one and was not elevated above the many long term contributors on the project. A hundred(s) billion dollar ecosystem shouldn't depend on the whims of a small group of maintainers who refuse to engage on decisions that not only impact the project but potentially the entire ecosystem (currently valued in the hundreds of billions of dollars). The fa
...
💬 dergoegge commented on pull request "p2p: Avoid prematurely clearing download state for other peers":
(https://github.com/bitcoin/bitcoin/pull/27608#issuecomment-1543775838)
post-merge Code review ACK 52e52071e01f4e98d87a47e1d5f3c5c3cc6dbaf4
(https://github.com/bitcoin/bitcoin/pull/27608#issuecomment-1543775838)
post-merge Code review ACK 52e52071e01f4e98d87a47e1d5f3c5c3cc6dbaf4
💬 stickies-v commented on pull request "add ryanofsky to trusted-keys":
(https://github.com/bitcoin/bitcoin/pull/27604#issuecomment-1543798949)
> The fact that I'm the only one here stating the obvious (...) is really scary to me.
Not to speak for anyone else, but perhaps the lack of engagement on your messages is because most regular contributors are just generally happy with the maintainer decisions and don't agree with your assessment but do not wish to keep discussing the same thing time and again?
(https://github.com/bitcoin/bitcoin/pull/27604#issuecomment-1543798949)
> The fact that I'm the only one here stating the obvious (...) is really scary to me.
Not to speak for anyone else, but perhaps the lack of engagement on your messages is because most regular contributors are just generally happy with the maintainer decisions and don't agree with your assessment but do not wish to keep discussing the same thing time and again?