📝 furszy opened a pull request: "[WIP] descriptors: do not return top-level only funcs as sub descriptors"
(https://github.com/bitcoin/bitcoin/pull/28067)
Linked to #28057.
Under WIP because I'm still exploring the topic. But pushing it to gather more opinions and insights.
Currently, the `InferScript` function returns an invalid descriptor when it tries to infer a p2sh-p2pkh script whose pubkey is not known by the wallet.
This behavior occurs because the inference process bypasses the `pkh` subscript when the pubkey is not contained by the wallet (no pubkey provider), interpreting it as a `sh(addr(ADDR))` descriptor. Then, the failure aris
...
(https://github.com/bitcoin/bitcoin/pull/28067)
Linked to #28057.
Under WIP because I'm still exploring the topic. But pushing it to gather more opinions and insights.
Currently, the `InferScript` function returns an invalid descriptor when it tries to infer a p2sh-p2pkh script whose pubkey is not known by the wallet.
This behavior occurs because the inference process bypasses the `pkh` subscript when the pubkey is not contained by the wallet (no pubkey provider), interpreting it as a `sh(addr(ADDR))` descriptor. Then, the failure aris
...
💬 MarcoFalke commented on pull request "[WIP] descriptors: do not return top-level only funcs as sub descriptors":
(https://github.com/bitcoin/bitcoin/pull/28067#issuecomment-1631097368)
> For reviewers, would recommend to start by examining the functional test to understand the context and the circumstances on which this can result in a fatal error (e.g. during the migration process).
The added functional test doesn't crash, no?
(https://github.com/bitcoin/bitcoin/pull/28067#issuecomment-1631097368)
> For reviewers, would recommend to start by examining the functional test to understand the context and the circumstances on which this can result in a fatal error (e.g. during the migration process).
The added functional test doesn't crash, no?
👍 jamesob approved a pull request: "test: indexes, fix on error infinite loop"
(https://github.com/bitcoin/bitcoin/pull/28044#pullrequestreview-1524633487)
ACK 89ba890
Built/tested locally. Seems like a commonsense way to assert-fail instead of causing an infinite loop on CI machines.
(https://github.com/bitcoin/bitcoin/pull/28044#pullrequestreview-1524633487)
ACK 89ba890
Built/tested locally. Seems like a commonsense way to assert-fail instead of causing an infinite loop on CI machines.
💬 MarcoFalke commented on pull request "test: indexes, fix on error infinite loop":
(https://github.com/bitcoin/bitcoin/pull/28044#issuecomment-1631104387)
lgtm ACK 89ba8905f5c68ae29412f9c4010314c5a113c234
(https://github.com/bitcoin/bitcoin/pull/28044#issuecomment-1631104387)
lgtm ACK 89ba8905f5c68ae29412f9c4010314c5a113c234
👍 ryanofsky approved a pull request: "test: indexes, fix on error infinite loop"
(https://github.com/bitcoin/bitcoin/pull/28044#pullrequestreview-1524643965)
Code review ACK 89ba8905f5c68ae29412f9c4010314c5a113c234. Just comment update since last review
(https://github.com/bitcoin/bitcoin/pull/28044#pullrequestreview-1524643965)
Code review ACK 89ba8905f5c68ae29412f9c4010314c5a113c234. Just comment update since last review
💬 brunoerg commented on pull request "fuzz: Generate process_message targets individually":
(https://github.com/bitcoin/bitcoin/pull/28066#discussion_r1259974561)
I think `--perl-regexp` won't work on MacOS.
See (MacOS 13.0 (M1)):
```sh
➜ bitcoin-core-dev git:(28066-marco) ✗ ./test/fuzz/test_runner.py corpus process_message -g
1 of 168 detected fuzz target(s) selected: process_message
Generating corpus to corpus
grep: unrecognized option `--perl-regexp'
usage: grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action
...
(https://github.com/bitcoin/bitcoin/pull/28066#discussion_r1259974561)
I think `--perl-regexp` won't work on MacOS.
See (MacOS 13.0 (M1)):
```sh
➜ bitcoin-core-dev git:(28066-marco) ✗ ./test/fuzz/test_runner.py corpus process_message -g
1 of 168 detected fuzz target(s) selected: process_message
Generating corpus to corpus
grep: unrecognized option `--perl-regexp'
usage: grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action
...
🤔 vasild reviewed a pull request: "net: disconnect inside AttemptToEvictConnection"
(https://github.com/bitcoin/bitcoin/pull/27912#pullrequestreview-1524221824)
I am not sure yet what would be the best approach to resolve the issues below.
One way would be to hold `m_nodes_disconnected_mutex` for the entire iteration of the `m_nodes_disconnected` list. But this would mean to call `DeleteNode()` and thus `PeerManagerImpl::FinalizeNode()` under that mutex. The latter locks `cs_main` :-(
(https://github.com/bitcoin/bitcoin/pull/27912#pullrequestreview-1524221824)
I am not sure yet what would be the best approach to resolve the issues below.
One way would be to hold `m_nodes_disconnected_mutex` for the entire iteration of the `m_nodes_disconnected` list. But this would mean to call `DeleteNode()` and thus `PeerManagerImpl::FinalizeNode()` under that mutex. The latter locks `cs_main` :-(
💬 vasild commented on pull request "net: disconnect inside AttemptToEvictConnection":
(https://github.com/bitcoin/bitcoin/pull/27912#discussion_r1259711622)
This can end up with a double-free if two threads concurrently execute it.
1. Thread1 makes a copy of `m_nodes_disconnected` and releases `m_nodes_disconnected_mutex`
2. Thread2 does the same
3. Thread1 starts iterating on its own copy and calls `DeleteDisconnectedNode()` on the first element which calls `DeleteNode()` which calls `delete pnode;`
4. Thread2 does the same on its own copy, a second `delete` on the same `CNode` object.
(https://github.com/bitcoin/bitcoin/pull/27912#discussion_r1259711622)
This can end up with a double-free if two threads concurrently execute it.
1. Thread1 makes a copy of `m_nodes_disconnected` and releases `m_nodes_disconnected_mutex`
2. Thread2 does the same
3. Thread1 starts iterating on its own copy and calls `DeleteDisconnectedNode()` on the first element which calls `DeleteNode()` which calls `delete pnode;`
4. Thread2 does the same on its own copy, a second `delete` on the same `CNode` object.
💬 vasild commented on pull request "net: disconnect inside AttemptToEvictConnection":
(https://github.com/bitcoin/bitcoin/pull/27912#discussion_r1259704670)
This need not be `RecursiveMutex`?
```suggestion
mutable Mutex m_nodes_disconnected_mutex;
```
(https://github.com/bitcoin/bitcoin/pull/27912#discussion_r1259704670)
This need not be `RecursiveMutex`?
```suggestion
mutable Mutex m_nodes_disconnected_mutex;
```
💬 vasild commented on pull request "net: disconnect inside AttemptToEvictConnection":
(https://github.com/bitcoin/bitcoin/pull/27912#discussion_r1259898235)
This could be called concurrently by two threads for the same `CNode`.
(https://github.com/bitcoin/bitcoin/pull/27912#discussion_r1259898235)
This could be called concurrently by two threads for the same `CNode`.
💬 vasild commented on pull request "net: disconnect inside AttemptToEvictConnection":
(https://github.com/bitcoin/bitcoin/pull/27912#discussion_r1259736855)
Same double-free as above (even though when this code in `StopNodes()` is executed, then the other threads that could access `m_nodes_disconnected` should have been exited by `StopThreads()` already, but better not rely on that).
(https://github.com/bitcoin/bitcoin/pull/27912#discussion_r1259736855)
Same double-free as above (even though when this code in `StopNodes()` is executed, then the other threads that could access `m_nodes_disconnected` should have been exited by `StopThreads()` already, but better not rely on that).
💬 furszy commented on pull request "[WIP] descriptors: do not return top-level only funcs as sub descriptors":
(https://github.com/bitcoin/bitcoin/pull/28067#issuecomment-1631128450)
> > For reviewers, would recommend to start by examining the functional test to understand the context and the circumstances on which this can result in a fatal error (e.g. during the migration process).
>
> The added functional test doesn't crash, does it?
Yeah. Without the fix commit, the migration process in the test will pass "successfully" and then, at the wallet restart verification, the wallet loading process will throw the "Unrecognized descriptor found" error. Which denotes that t
...
(https://github.com/bitcoin/bitcoin/pull/28067#issuecomment-1631128450)
> > For reviewers, would recommend to start by examining the functional test to understand the context and the circumstances on which this can result in a fatal error (e.g. during the migration process).
>
> The added functional test doesn't crash, does it?
Yeah. Without the fix commit, the migration process in the test will pass "successfully" and then, at the wallet restart verification, the wallet loading process will throw the "Unrecognized descriptor found" error. Which denotes that t
...
💬 sipa commented on pull request "[WIP] descriptors: do not return top-level only funcs as sub descriptors":
(https://github.com/bitcoin/bitcoin/pull/28067#issuecomment-1631134852)
This is clearly an issue; we should not be inferring descriptors that the code itself doesn't accept back.
On the other hand, maybe we should just permit `addr()` and `raw()` inside sh/wsh/tr. There is discussion about that in #24114 too.
(https://github.com/bitcoin/bitcoin/pull/28067#issuecomment-1631134852)
This is clearly an issue; we should not be inferring descriptors that the code itself doesn't accept back.
On the other hand, maybe we should just permit `addr()` and `raw()` inside sh/wsh/tr. There is discussion about that in #24114 too.
🚀 ryanofsky merged a pull request: "test: indexes, fix on error infinite loop"
(https://github.com/bitcoin/bitcoin/pull/28044)
(https://github.com/bitcoin/bitcoin/pull/28044)
💬 sipa commented on pull request "[WIP] descriptors: do not return top-level only funcs as sub descriptors":
(https://github.com/bitcoin/bitcoin/pull/28067#discussion_r1260003229)
Coding style nit: if you have more than a single line `if` statement, you must use braces and indentation.
(https://github.com/bitcoin/bitcoin/pull/28067#discussion_r1260003229)
Coding style nit: if you have more than a single line `if` statement, you must use braces and indentation.
💬 furszy commented on pull request "[WIP] descriptors: do not return top-level only funcs as sub descriptors":
(https://github.com/bitcoin/bitcoin/pull/28067#issuecomment-1631222345)
> On the other hand, maybe we should just permit `addr()` and `raw()` inside sh/wsh/tr. There is discussion about that in #24114 too.
Nice, I was asking myself the same question.
I actually started implementing this as a custom, sort of dummy, pubkey provider which had no knowledge of its keys, only containing the key ids, inside the `PKHDescriptor`. So it mapped to the original `sh(pkh(key_id))`. But.. I ended up preferring this approach for the "controversy" that the other one could had.
...
(https://github.com/bitcoin/bitcoin/pull/28067#issuecomment-1631222345)
> On the other hand, maybe we should just permit `addr()` and `raw()` inside sh/wsh/tr. There is discussion about that in #24114 too.
Nice, I was asking myself the same question.
I actually started implementing this as a custom, sort of dummy, pubkey provider which had no knowledge of its keys, only containing the key ids, inside the `PKHDescriptor`. So it mapped to the original `sh(pkh(key_id))`. But.. I ended up preferring this approach for the "controversy" that the other one could had.
...
💬 MarcoFalke commented on pull request "fuzz: Generate process_message targets individually":
(https://github.com/bitcoin/bitcoin/pull/28066#discussion_r1260062800)
thx, fixed
(https://github.com/bitcoin/bitcoin/pull/28066#discussion_r1260062800)
thx, fixed
🤔 mzumsande reviewed a pull request: "kernel: Remove StartShutdown calls from validation code"
(https://github.com/bitcoin/bitcoin/pull/28048#pullrequestreview-1524806674)
Concept ACK
(https://github.com/bitcoin/bitcoin/pull/28048#pullrequestreview-1524806674)
Concept ACK
💬 mzumsande commented on pull request "kernel: Remove StartShutdown calls from validation code":
(https://github.com/bitcoin/bitcoin/pull/28048#discussion_r1260065112)
Just wanted to note that this proposal has some history, see Issue #13477 and the two closed PRs: #13490 and #13713.
In particular, comment https://github.com/bitcoin/bitcoin/pull/13490#issuecomment-398069237 still seems relevant:
The expectation that -stopatheight stops exactly at the specified height relies on `ActivateBestChainStep()` only connecting a single block in each invocation, which is currently the case, but more
of a coincidence due to not wanting to lock `cs_main` for too lon
...
(https://github.com/bitcoin/bitcoin/pull/28048#discussion_r1260065112)
Just wanted to note that this proposal has some history, see Issue #13477 and the two closed PRs: #13490 and #13713.
In particular, comment https://github.com/bitcoin/bitcoin/pull/13490#issuecomment-398069237 still seems relevant:
The expectation that -stopatheight stops exactly at the specified height relies on `ActivateBestChainStep()` only connecting a single block in each invocation, which is currently the case, but more
of a coincidence due to not wanting to lock `cs_main` for too lon
...
🤔 MarcoFalke reviewed a pull request: "Add support for "partial" fuzzers that indicate usefulness"
(https://github.com/bitcoin/bitcoin/pull/27552#pullrequestreview-1524835460)
lgtm, but would be good to test this before merge
(https://github.com/bitcoin/bitcoin/pull/27552#pullrequestreview-1524835460)
lgtm, but would be good to test this before merge