Skip to content

A CodeChain node keeps emitting "sendError" #1931

Open
Open
A CodeChain node keeps emitting "sendError"#1931
Labels
bugSomething isn't working

Description

@seonpyoKim
seonpyoKim
opened on Dec 17, 2019

This phenomenon was found on 2019.12.17 at about 2 P.M.
The monitor sends a codechain death confirmation because the node monitor that was watching had been dead. The log is like the following:

#364967762 2019-12-17 05:37:18.685152917 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:19.224282592 UTC P2P Worker #1 WARN netapi  block-propagation cannot message Global 34.85.31.119:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:19.355859641 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:19.553804053 UTC P2P Worker #1 WARN netapi  block-propagation cannot message Global 35.230.2.230:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:19.590484103 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:19.638622014 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 35.233.0.162:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:19.642450809 UTC P2P Worker #0 WARN netapi  block-propagation cannot message Global 34.85.92.6:3485: "SendError(..)"                                             
#364967762 2019-12-17 05:37:19.685322323 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:20.593226243 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:20.685409734 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:21.548407383 UTC P2P Worker #3 WARN netapi  block-propagation cannot message Global 35.230.2.230:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:21.595849899 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:21.614647820 UTC P2P Worker #0 WARN netapi  block-propagation cannot message Global 35.233.0.162:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:21.641915189 UTC P2P Worker #3 WARN netapi  block-propagation cannot message Global 34.85.31.119:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:21.685501929 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:22.059731325 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 34.85.92.6:3485: "SendError(..)"                                             
#364967762 2019-12-17 05:37:22.357984173 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:22.598637390 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:22.685612250 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:23.086173404 UTC P2P Worker #1 WARN netapi  block-propagation cannot message Global 34.85.31.119:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:23.211959453 UTC P2P Worker #3 WARN netapi  block-propagation cannot message Global 35.230.2.230:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:23.420002722 UTC P2P Worker #1 WARN netapi  block-propagation cannot message Global 34.85.92.6:3485: "SendError(..)"                                             
#364967762 2019-12-17 05:37:23.601295048 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:23.641766807 UTC P2P Worker #3 WARN netapi  block-propagation cannot message Global 35.233.0.162:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:23.685702369 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:24.604211465 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:24.685837824 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:25.183168256 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 35.230.2.230:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:25.344193384 UTC P2P Worker #1 WARN netapi  block-propagation cannot message Global 34.85.31.119:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:25.359482897 UTC P2P Worker #0 WARN netapi  block-propagation cannot message Global 35.233.0.162:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:25.361471308 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:25.391840906 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 34.85.92.6:3485: "SendError(..)"                                             
#364967762 2019-12-17 05:37:25.606731117 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:25.685960279 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:26.609702715 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:26.686112553 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:26.860126470 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 34.85.92.6:3485: "SendError(..)"                                             
#364967762 2019-12-17 05:37:26.861161222 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 34.85.31.119:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:26.929697545 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 35.230.2.230:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:26.937245993 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 35.233.0.162:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:27.612464839 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:27.686310331 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"                                                                 
#364967762 2019-12-17 05:37:28.364441429 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:28.615200476 UTC http.worker0 INFO rpc  RPC call(chain_getBestBlockNumber([]))                                                                                   
#364967762 2019-12-17 05:37:28.658031061 UTC P2P Worker #0 WARN netapi  block-propagation cannot message Global 34.85.31.119:3485: "SendError(..)"                                           
#364967762 2019-12-17 05:37:28.686393587 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"

Activity

  1. majecty
    added
    bugSomething isn't working
    on Dec 17, 2019

  2. MSNTCS commented on Apr 1, 2020

    @MSNTCS
    MSNTCS
    on Apr 1, 2020
    Contributor

    The same phenomena happened again at 2020.04.01 at 11:45 A.M.
    Here is the log:

    #14484478 2020-04-01 05:30:54.968956978 UTC http.worker00 INFO rpc  RPC call(chain_getBestBlockNumber([]))
#14484478 2020-04-01 05:30:54.969080439 UTC http.worker00 INFO rpc  RPC call(chain_getBestBlockNumber([]))
#14484478 2020-04-01 05:30:55.161124662 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 35.196.235.61:3485: "SendError(..)"
#14484478 2020-04-01 05:30:55.163761379 UTC P2P Worker #0 WARN netapi  block-propagation cannot message Global 35.196.159.218:3485: "SendError(..)"
#14484478 2020-04-01 05:30:55.341143075 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 35.241.238.157:3485: "SendError(..)"
#14484478 2020-04-01 05:30:55.360265253 UTC P2P Worker #1 WARN netapi  block-propagation cannot message Global 35.240.69.133:3485: "SendError(..)"
#14484478 2020-04-01 05:30:55.681324878 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"
#14484478 2020-04-01 05:30:55.971981974 UTC http.worker00 INFO rpc  RPC call(chain_getBestBlockNumber([]))
#14484478 2020-04-01 05:30:55.972121571 UTC http.worker00 INFO rpc  RPC call(chain_getBestBlockNumber([]))
#14484478 2020-04-01 05:30:56.573386962 UTC http.worker00 INFO rpc  RPC call(chain_getBestBlockNumber([]))
#14484478 2020-04-01 05:30:56.573528258 UTC http.worker00 INFO rpc  RPC call(chain_getBestBlockNumber([]))
#14484478 2020-04-01 05:30:56.681465348 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"
#14484478 2020-04-01 05:30:56.899170269 UTC P2P Worker #1 WARN netapi  block-propagation cannot message Global 34.87.122.166:3485: "SendError(..)"
#14484478 2020-04-01 05:30:56.974871068 UTC http.worker00 INFO rpc  RPC call(chain_getBestBlockNumber([]))
#14484478 2020-04-01 05:30:56.975084573 UTC http.worker00 INFO rpc  RPC call(chain_getBestBlockNumber([]))
#14484478 2020-04-01 05:30:57.291022944 UTC P2P Worker #0 WARN netapi  block-propagation cannot message Global 35.196.159.218:3485: "SendError(..)"
#14484478 2020-04-01 05:30:57.306459133 UTC P2P Worker #2 WARN netapi  block-propagation cannot message Global 35.196.235.61:3485: "SendError(..)"
#14484478 2020-04-01 05:30:57.350251572 UTC P2P Worker #3 WARN netapi  block-propagation cannot message Global 35.241.238.157:3485: "SendError(..)"
#14484478 2020-04-01 05:30:57.356194430 UTC P2P Worker #0 WARN netapi  block-propagation cannot message Global 35.240.69.133:3485: "SendError(..)"
^C#14484478 2020-04-01 05:30:57.681617332 UTC timer.worker.1 WARN netapi  block-propagation cannot timeout 0: "SendError(..)"```

  3. majecty commented on Apr 1, 2020

    @majecty
    majecty
    on Apr 1, 2020

    I checked the code. The "SendError" means that extension threads were killed. We need to know why the threads were killed. We need log messages before the "SendError" to find the reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions