You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Aug 24, 2020. It is now read-only.
Repository navigation
This repository was archived by the owner on Aug 24, 2020. It is now read-only.
Use service discovery in scheduler to discover where log-cache nodes are #52
@apoydence I was thinking of doing something similar to what the loggregator agent does.
Accept a DNS name in config.
At runtime query DNS to get A or AAAA records for that name.
Use these IP addresses.
Periodically query DNS to refresh the list of IP addresses.
This would be compatible with a LOT of service discovery systems, including kube-dns and bosh-dns.
Alternatively, you can query for SRV records which allows for port, weighting, transport and service name to be discovered. kube-dns supports this but I am not sure about bosh-dns.
I would want to do more experiments with multiple schedulers after this is completed to see if there is any thrashing. I suspect each scheduler would be looking at a different subset of log cache nodes and would be instructing them differently.
That is a possibility. We can have an algo that is resistant to sudden changes to prevent nodes dropping out when they come right back. Something like a TTL for a node to stop being scheduled to. This would help with thrashing, allow for nodes that are truely gone to expire, and allow for new nodes to come online.
Activity
cf-gitbot commentedon Apr 30, 2018
We have created an issue in Pivotal Tracker to manage this:
https://www.pivotaltracker.com/story/show/157208717
The labels on this github issue will be updated when the story is started.
poy commentedon May 16, 2018
@jasonkeene How do other systems manage this? Some form of a gossip protocol?
jasonkeene commentedon May 16, 2018
@apoydence I was thinking of doing something similar to what the loggregator agent does.
This would be compatible with a LOT of service discovery systems, including kube-dns and bosh-dns.
Alternatively, you can query for SRV records which allows for port, weighting, transport and service name to be discovered. kube-dns supports this but I am not sure about bosh-dns.
poy commentedon May 22, 2018
@jasonkeene This sounds reasonable.
I would want to do more experiments with multiple schedulers after this is completed to see if there is any thrashing. I suspect each scheduler would be looking at a different subset of log cache nodes and would be instructing them differently.
jasonkeene commentedon May 22, 2018
That is a possibility. We can have an algo that is resistant to sudden changes to prevent nodes dropping out when they come right back. Something like a TTL for a node to stop being scheduled to. This would help with thrashing, allow for nodes that are truely gone to expire, and allow for new nodes to come online.