💬 hebasto commented on pull request "Revert "contrib: macdeploy: monkey-patch gen-sdk to be deterministic"":
(https://github.com/bitcoin/bitcoin/pull/30282#issuecomment-2171272886)
cc @prusnak
(https://github.com/bitcoin/bitcoin/pull/30282#issuecomment-2171272886)
cc @prusnak
👍 hebasto approved a pull request: "macOS: rewrite some docs & swap `mmacosx-version-min` for `mmacos-version-min`"
(https://github.com/bitcoin/bitcoin/pull/30287#pullrequestreview-2121294059)
ACK 7c298fe0df38696f60e981469422315c03a722da.
My Guix builds:
```
x86_64
c749e06e582abccb481fed3b1754f209d81a7092840bfe18ee68a68b1cafef0d guix-build-7c298fe0df38/output/arm64-apple-darwin/SHA256SUMS.part
5b0d3d772146694214c6041c299efb05c2f76c9fe7dc80fc522270786f426388 guix-build-7c298fe0df38/output/arm64-apple-darwin/bitcoin-7c298fe0df38-arm64-apple-darwin-unsigned.tar.gz
537b5e3068e4939c36b758dd8957f25ffa66b572663e9bc699ce109857aa1e6a guix-build-7c298fe0df38/output/arm64-apple-darwin
...
(https://github.com/bitcoin/bitcoin/pull/30287#pullrequestreview-2121294059)
ACK 7c298fe0df38696f60e981469422315c03a722da.
My Guix builds:
```
x86_64
c749e06e582abccb481fed3b1754f209d81a7092840bfe18ee68a68b1cafef0d guix-build-7c298fe0df38/output/arm64-apple-darwin/SHA256SUMS.part
5b0d3d772146694214c6041c299efb05c2f76c9fe7dc80fc522270786f426388 guix-build-7c298fe0df38/output/arm64-apple-darwin/bitcoin-7c298fe0df38-arm64-apple-darwin-unsigned.tar.gz
537b5e3068e4939c36b758dd8957f25ffa66b572663e9bc699ce109857aa1e6a guix-build-7c298fe0df38/output/arm64-apple-darwin
...
📝 hMsats opened a pull request: "Always show 100% verification progress when done"
(https://github.com/bitcoin/bitcoin/pull/30293)
External software/scripts may monitor Bitcoin Core's verification progress in debug.log.
Now these have to scan for "No coin database inconsistencies" to see if the verification progress has finished. It's cleaner to always show 100% when the verification progress has finished (without encountering any problems). If I remember correctly this was the case in older versions of Bitcoin Core.
NOW:
```
2024-06-16T09:12:03Z Verification progress: 0%
2024-06-16T09:12:55Z Verification progress:
...
(https://github.com/bitcoin/bitcoin/pull/30293)
External software/scripts may monitor Bitcoin Core's verification progress in debug.log.
Now these have to scan for "No coin database inconsistencies" to see if the verification progress has finished. It's cleaner to always show 100% when the verification progress has finished (without encountering any problems). If I remember correctly this was the case in older versions of Bitcoin Core.
NOW:
```
2024-06-16T09:12:03Z Verification progress: 0%
2024-06-16T09:12:55Z Verification progress:
...
💬 pinheadmz commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171512300)
> External software/scripts may monitor Bitcoin Core's verification progress in debug.log.
Do you run a script like this? I'm curious what the motivation is, since I'm pretty sure if verification fails the program quits.
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171512300)
> External software/scripts may monitor Bitcoin Core's verification progress in debug.log.
Do you run a script like this? I'm curious what the motivation is, since I'm pretty sure if verification fails the program quits.
💬 ryanofsky commented on issue "RFC: Assumeutxo and large forks and reorgs":
(https://github.com/bitcoin/bitcoin/issues/30288#issuecomment-2171582899)
If I can summarize and clarify, neither of you think the current behavior of locking in snapshot block, and temporarily refusing to consider chains that don't include it is a good idea? The list of reasons above trying to justify current behavior are basically B.S.? (The "Possible advantages of original chainstate targeting the snapshot block" section about network hard forks and eclipse attacks)
Instead, the behavior you both seem to prefer is just: when a snapshot is loaded, as soon as a ne
...
(https://github.com/bitcoin/bitcoin/issues/30288#issuecomment-2171582899)
If I can summarize and clarify, neither of you think the current behavior of locking in snapshot block, and temporarily refusing to consider chains that don't include it is a good idea? The list of reasons above trying to justify current behavior are basically B.S.? (The "Possible advantages of original chainstate targeting the snapshot block" section about network hard forks and eclipse attacks)
Instead, the behavior you both seem to prefer is just: when a snapshot is loaded, as soon as a ne
...
💬 hMsats commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171583802)
@pinheadmz yes, I make a system backup every month like this: pre-backup with rsync, turn off my bitcoin node, quick final rsync, restart bitcoin node and wait for Bitcoin Core to finish (while showing verification progress) before starting my public electrum server, lightning node and pruning node (only connected to my main node).
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171583802)
@pinheadmz yes, I make a system backup every month like this: pre-backup with rsync, turn off my bitcoin node, quick final rsync, restart bitcoin node and wait for Bitcoin Core to finish (while showing verification progress) before starting my public electrum server, lightning node and pruning node (only connected to my main node).
💬 pinheadmz commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171615874)
Ok and do you understand that the verification progress you are touching here is in the startup sequence where Bitcoin ensures the database is consistent before even connecting to the network?
It's possible you may be referring to sync progress which is the process of downloading and verifying new blocks until the most work chain has been processed. That progress will never reach 100% by design. There is some relevant discussion here:
https://github.com/bitcoin/bitcoin/issues/28847
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171615874)
Ok and do you understand that the verification progress you are touching here is in the startup sequence where Bitcoin ensures the database is consistent before even connecting to the network?
It's possible you may be referring to sync progress which is the process of downloading and verifying new blocks until the most work chain has been processed. That progress will never reach 100% by design. There is some relevant discussion here:
https://github.com/bitcoin/bitcoin/issues/28847
🤔 tdb3 reviewed a pull request: "Always show 100% verification progress when done"
(https://github.com/bitcoin/bitcoin/pull/30293#pullrequestreview-2121441221)
Thanks for taking a look and presenting the idea.
With bitcoin providing an RPC interface (and the OS providing general process run state), sifting through the debug.log seems like a non-ideal method for status gathering. Are there some examples from out in the wild where this is relied upon?
A cool feature of the RPC interface is that it provides early feedback (e.g. that blocks are being verified). Here are some screenshots of what the RPC caller sees and what is occurring in the debug
...
(https://github.com/bitcoin/bitcoin/pull/30293#pullrequestreview-2121441221)
Thanks for taking a look and presenting the idea.
With bitcoin providing an RPC interface (and the OS providing general process run state), sifting through the debug.log seems like a non-ideal method for status gathering. Are there some examples from out in the wild where this is relied upon?
A cool feature of the RPC interface is that it provides early feedback (e.g. that blocks are being verified). Here are some screenshots of what the RPC caller sees and what is occurring in the debug
...
💬 hMsats commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171658767)
@tdb3
> There might be value in providing a definitive 100% message, but more for general accuracy to bitcoin itself rather than to machine observers of the debug.log.
That was the real reason for this pull request. My example was just a side note. Seeing the 100% is useful also for users looking at the `debug.log`.
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171658767)
@tdb3
> There might be value in providing a definitive 100% message, but more for general accuracy to bitcoin itself rather than to machine observers of the debug.log.
That was the real reason for this pull request. My example was just a side note. Seeing the 100% is useful also for users looking at the `debug.log`.
💬 hMsats commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171677111)
@pinheadmz Ah, maybe we need different strings here. Instead of "Verification progress" maybe "Validation progress" in order not to confuse the two processes.
Linux command: `ag "Verification progress"` gives (including the 100%):
```
bitcoin/test/functional/interface_bitcoin_cli.py
184: assert_equal(cli_get_info['Verification progress'], "%.4f%%" % (blockchain_info['verificationprogress'] * 100))
bitcoin/src/validation.cpp
4613: LogPrintf("Verification progress: 0%%\n");
...
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171677111)
@pinheadmz Ah, maybe we need different strings here. Instead of "Verification progress" maybe "Validation progress" in order not to confuse the two processes.
Linux command: `ag "Verification progress"` gives (including the 100%):
```
bitcoin/test/functional/interface_bitcoin_cli.py
184: assert_equal(cli_get_info['Verification progress'], "%.4f%%" % (blockchain_info['verificationprogress'] * 100))
bitcoin/src/validation.cpp
4613: LogPrintf("Verification progress: 0%%\n");
...
💬 Prabhat1308 commented on pull request "p2p: Fill reconciliation sets (Erlay) attempt 2":
(https://github.com/bitcoin/bitcoin/pull/30116#discussion_r1641905080)
The number of parents per peer should mostly be in the range of (0-2/3) , This should include a significant numbers of peers to reconcile with . A constant number might not be ideal considering if the array returned is smaller than that constant number , we might be reconciling with non-existent peers .
(https://github.com/bitcoin/bitcoin/pull/30116#discussion_r1641905080)
The number of parents per peer should mostly be in the range of (0-2/3) , This should include a significant numbers of peers to reconcile with . A constant number might not be ideal considering if the array returned is smaller than that constant number , we might be reconciling with non-existent peers .
💬 pinheadmz commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171746348)
> wait for Bitcoin Core to finish
At this point in your script do you mean
- "wait for Bitcoin Core to be ready for RPC commands" or
- "wait for Bitcoin Core to sync to the most work chain it can find on the network so my UTXO state is most likely to be globally accurate"?
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171746348)
> wait for Bitcoin Core to finish
At this point in your script do you mean
- "wait for Bitcoin Core to be ready for RPC commands" or
- "wait for Bitcoin Core to sync to the most work chain it can find on the network so my UTXO state is most likely to be globally accurate"?
💬 hMsats commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171750527)
@pinheadmz Irrespective of my (simple) shell script, in my opinion Bitcoin Core should always print "Verification progress=100%" or even better "Validation progress=100%" in `debug.log`.
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171750527)
@pinheadmz Irrespective of my (simple) shell script, in my opinion Bitcoin Core should always print "Verification progress=100%" or even better "Validation progress=100%" in `debug.log`.
💬 kevkevinpal commented on pull request "test: Added test coverage to listsinceblock rpc":
(https://github.com/bitcoin/bitcoin/pull/30195#issuecomment-2171781426)
removed an assertion that didn't seem needed in [881724d](https://github.com/bitcoin/bitcoin/pull/30195/commits/881724d443d11f984a721ef1edd5777c24d1ed29)
The other two seem fine as they assert the file was replaced properly which `.replace` would not throw an error for
(https://github.com/bitcoin/bitcoin/pull/30195#issuecomment-2171781426)
removed an assertion that didn't seem needed in [881724d](https://github.com/bitcoin/bitcoin/pull/30195/commits/881724d443d11f984a721ef1edd5777c24d1ed29)
The other two seem fine as they assert the file was replaced properly which `.replace` would not throw an error for
💬 achow101 commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171781775)
If you want to know when bitcoind is ready to start receiving RPCs and generally finished its startup sequence, you should use `-startupnotify` and have it notify you, rather than parsing the debug.log.
ISTM the log line saying that no inconsistencies were found is a good enough indicator that verification was finished.
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171781775)
If you want to know when bitcoind is ready to start receiving RPCs and generally finished its startup sequence, you should use `-startupnotify` and have it notify you, rather than parsing the debug.log.
ISTM the log line saying that no inconsistencies were found is a good enough indicator that verification was finished.
💬 kevkevinpal commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171793485)
> That progress will never reach 100% by design
I agree with @pinheadmz
why not just look for this line instead for your script?
`Verification: No coin database inconsistencies`
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171793485)
> That progress will never reach 100% by design
I agree with @pinheadmz
why not just look for this line instead for your script?
`Verification: No coin database inconsistencies`
💬 sdaftuar commented on issue "RFC: Assumeutxo and large forks and reorgs":
(https://github.com/bitcoin/bitcoin/issues/30288#issuecomment-2171794670)
> If I can summarize and clarify, neither of you think the current behavior of locking in snapshot block, and temporarily refusing to consider chains that don't include it is a good idea?
I'm not sure I'm following this point exactly: my recollection is that the current **observable** behavior in this scenario would be to crash, because even though the original/fully validated chainstate locks in ancestors of the snapshot block to be possible tips, we'd still try to reorg the snapshot chains
...
(https://github.com/bitcoin/bitcoin/issues/30288#issuecomment-2171794670)
> If I can summarize and clarify, neither of you think the current behavior of locking in snapshot block, and temporarily refusing to consider chains that don't include it is a good idea?
I'm not sure I'm following this point exactly: my recollection is that the current **observable** behavior in this scenario would be to crash, because even though the original/fully validated chainstate locks in ancestors of the snapshot block to be possible tips, we'd still try to reorg the snapshot chains
...
💬 kevkevinpal commented on pull request "test: write functional test results to csv":
(https://github.com/bitcoin/bitcoin/pull/30291#issuecomment-2171794835)
This line initially when I read it made me assume that 28 tests failed when in reality 28 passed and one failed
`ALL,Failed,28`
(https://github.com/bitcoin/bitcoin/pull/30291#issuecomment-2171794835)
This line initially when I read it made me assume that 28 tests failed when in reality 28 passed and one failed
`ALL,Failed,28`
💬 kevkevinpal commented on pull request "test: write functional test results to csv":
(https://github.com/bitcoin/bitcoin/pull/30291#discussion_r1641941956)
Would be useful to have a log come from this assert
```suggestion
assert results_filepath.parent.exists(), "Results file path parent directory does not exist"
```
Before:
```
test/functional/test_runner.py --resultsfile ./fakedir/functional_test_results.csv feature_blocksdir
Temporary test directory at /tmp/test_runner_₿_🏃_20240616_141849
Traceback (most recent call last):
File "/home/kevkevin/DEVDIR/bitcoin/test/functional/test_runner.py", line 939, in <module>
main()
...
(https://github.com/bitcoin/bitcoin/pull/30291#discussion_r1641941956)
Would be useful to have a log come from this assert
```suggestion
assert results_filepath.parent.exists(), "Results file path parent directory does not exist"
```
Before:
```
test/functional/test_runner.py --resultsfile ./fakedir/functional_test_results.csv feature_blocksdir
Temporary test directory at /tmp/test_runner_₿_🏃_20240616_141849
Traceback (most recent call last):
File "/home/kevkevin/DEVDIR/bitcoin/test/functional/test_runner.py", line 939, in <module>
main()
...
💬 hMsats commented on pull request "Always show 100% verification progress when done":
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171803437)
@kevkevinpal (and @achow101) I do in my script but I still believe that the two lines:
```
2024-06-16T09:13:27Z Verification progress: 99%
2024-06-16T09:13:27Z Verification: No coin database inconsistencies in last 6 blocks (39935 transactions)
```
are ugly and
```
2024-06-16T09:13:27Z Verification progress: 99%
2024-06-16T09:13:27Z Verification progress: 100%
2024-06-16T09:13:27Z Verification: No coin database inconsistencies in last 6 blocks (39935 transactions)
```
is much bett
...
(https://github.com/bitcoin/bitcoin/pull/30293#issuecomment-2171803437)
@kevkevinpal (and @achow101) I do in my script but I still believe that the two lines:
```
2024-06-16T09:13:27Z Verification progress: 99%
2024-06-16T09:13:27Z Verification: No coin database inconsistencies in last 6 blocks (39935 transactions)
```
are ugly and
```
2024-06-16T09:13:27Z Verification progress: 99%
2024-06-16T09:13:27Z Verification progress: 100%
2024-06-16T09:13:27Z Verification: No coin database inconsistencies in last 6 blocks (39935 transactions)
```
is much bett
...