Repository navigation
Jenkins downtime #3709
Description
@nodejs/collaborators we have taken Jenkins down due to some security concerns, nothing major as far as we've been able to determine so far so this is just precautionary. We are in the process of doing some cleaning and will try and get it back online ASAP. We'll keep you updated here.
Activity
jbergstroem commentedon Nov 11, 2015
Update: We've now redeployed the master ci server and most of the slaves. Expect everything to be up and running 'as normal' in the coming 12 hours. I will close this issue when done and update ci documentation if needed.
We've also switched authentication strategy -- upon logging in you will authenticate through github. If you have any issues, ping the @nodejs/build team.
Fishrock123 commentedon Nov 11, 2015
@jbergstroem 👏👏👏Awesome!
@nodejs/build Thanks for all the hard work. :)
cjihrig commentedon Nov 11, 2015
Yes, thank you for all the hard work!
thefourtheye commentedon Nov 11, 2015
Great work :-) Apart from that, one less password to remember. Yay 👍
jbergstroem commentedon Nov 11, 2015
Ok, we should be back to normal now (99%). We still have some reprovisioning to do but it shouldn't affect too many things. Great job @nodejs/build crew.
rvagg commentedon Nov 14, 2015
I've just realised that we didn't properly explain what was going on, hopefully @nodejs/collaborators don't feel too in the dark about it, we initially kept it private so we could react calmly but there's no reason to be particularly secretive about it now. If you have further questions please ask.
Basically we were responding to this Jenkins security issue which is pretty severe. We have found no indication that our Jenkins server was messed with but we have taken a lot of precautionary actions including reprovisioning servers, setting up more restrictive firewall rules etc. We still have work to do and we've taken the opportunity to tweak our infrastructure, including the new GitHub authentication as well as removing some of the nodes and adding new ones.
rvagg commentedon Nov 14, 2015
@nodejs/collaborators Jenkins is (mostly) back to normal now and should be able to process your jobs, if you continue to have Jenkins problems please ping @nodejs/build.
@nodejs/release release builders should be good to go, and I'm also making excellent progress on <= 0.12 builds.