Last week was the annual Fermilab User’s Meeting, for all sorts of interesting talks see here. These included a talk by Sergio Bertolucci giving recent news about the LHC status. This week CERN is hosting a CERN-Fermilab Hadron Collider Physics summer school, talks here, including one from Jorg Wenninger about the LHC status.
The main concern now involves bad soldering of some of the 1700 or so bus-bar inter-connections between magnets. One of these seems to have been at the origin of the accident last September. For the sectors (four of them) that are warm, such bad joints can be identified relatively easily, by their higher-than-normal resistance, and repaired. Unfortunately, for the four sectors that are now cold, identifying such bad splices is much more difficult. Warming up these sectors and cooling them back down is a time-consuming process that could significantly push back the LHC schedule.
Late last week the decision was made to start warming up sector 45. Measurements at 80K had identified 3 cases of anomalously high resistance. The plan is to warm up the sector, take measurements which can be compared to the measurements made when the sector was cool, and fix splices as necessary. For this particular sector, things can be rearranged so that warming it up and cooling it back down will not seriously impact the schedule.
For the other three cold sectors though, the situation remains unclear. They’re gathering more data and analyzing it, trying to understand better what is going on, as well as analyzing the question of whether it’s possible to go ahead and find ways to run the machine safely, even given the possible existence of somewhat iffy interconnections.
The latest version of the schedule, from mid-last week, has powering tests ending in mid-October. So, as long as it does not turn out to be necessary to warm up more sectors, late October is the time-frame for trying again to circulate a beam and begin beam commissioning.
Update: There’s a video of Wenninger’s talk last week available here, where he gave some more details in the question session afterwards. Sector 45 will be warm and ready for measurements next Monday (June 22). If the results show good correlation with what was measured at 80K, at the end of the month the temperature of the three remaining sectors will be stabilized at 80K and measurements will be made on those.
About the current schedule for when to try and circulate a beam, he says “I know that officially it’s still September [last week] but I have problems to sell that…” with a better guess of sometime in October (assuming the three sectors at issue don’t need to be warmed up). He also remarks that it will be a while (2012?) if ever before the machine operates at 7 TeV. 5 TeV is the likely energy at the beginning, with a possibility of going up to 6 TeV.
Isn’t is surprising that such a straightforward test, i.e., the test for anomalously high resistance wasn’t part of the quality control procedure during the initial construction phase?
Makes you wonder.
I’m also surprised that it has taken so long to identify this issue. Had it been flagged earlier there would have been plenty of time to warm up the sectors, test and fix.
Perhaps there is something I’m missing.
>Perhaps there is something I’m missing.
You are probably underestimating the vagaries of real-world engineering. Especially for “once-off” projects of high complexity, things are left out, checks are not made, problems are not considered due to failure of imaginations, paperwork is lost and of course short-cuts are made due to budget and time constraints. Moreover, a lot of the stuff being designed is bleeding edge, so you cannot just expect production-run quality in any case.
How are these connections soldered? I am familiar with control & construction work from 5 VDC to 13000 VAC & know a good electrical tech can tell an “iffy” joint by just looking. Of course, a resistance test proves it.
I’m wondering if they could use some kind of isolated thermal suits to go inside the cooled down areas and do the repairs. If people can repair the Hubble space telescope in orbit I would hope this to be a simpler operation.
My understanding is that the problem is that there is not yet a reliable way of figuring out which of the cooled-down interconnection might be problematic and need repairs.
You’d think there would be a better way to do things by now. The idea of having to shut down the entire LHC for weeks to deal with even a slight error resembles a programming project in which the most minute change requires a complete recomplication. If the LHC fails again it’s going to look very bad for the entire project.
Whats with the cool down times anyway? How long are the times at other accelerators like the Tevatron?
These are not your ordinary solder joints! The interconnects are layered with alloys of platinum if I recall correctly. Modelling behavior as it is cooled may be a difficult problem.
As an aside, these are anxious times for any project that depends on quantities of PGMs. Take a look at the five year
chart for platinum:
>>The idea of having to shut down the entire LHC for weeks to deal with even a slight error resembles a programming project in which the most minute change requires a complete recomplication.
You meant to say “recompliation” I suppose (note that recompilations indeed took significant time a few decades hence, when I first hit the keyboards, but today are measured in minutes, which is nice as we then get “continuous recompilation” and the attendant quality improvements – but that’s another story).
Your comparison is wrong of course – I would say you could compare the innocent-sounding “solder joint fixing” with opening up, reviewing and redesigning a sizeable part of a software architecture for an embedded system which in this case is partly live. Any prospect of doing that will put cause sweat to appear on any project manager’s brow.
Ouch, I misspelled “recompilation”, too. I blame my lousy keyboard and the infuriatingly tiny blog text area!
I cannot now locate the info on PGM metal use in the LHC, other than the use of Platinum for the temperature sensors. Sorry for the mis-information.
But there are plenty of exotic alloys throughout the machine. In the interconnection assemblies alone there are some quarter million components!
The number of connections is given at 123,00: http://cerncourier.com/cws/article/cern/29012
A note from February CERN COURIER: Meanwhile, work continues apace on the repairs at the LHC. At the end of January, a dipole from sector 1-2, which had been identified as having an internal splice resistance of 100 nΩ, was opened up after removal from the tunnel and was found to have little solder on the splice joint. It is likely that a similar small resistance was at the root of the incident in sector 3-4. The LHC teams can now detect a single defective splice in situ when a sector is cold and they have identified another dipole showing a similar defect in sector 6-7. This sector will be warmed up and the magnet removed. Each sector has more than 2500 splices, but the resistance tests can only be conducted on cold magnets. Three sectors remain to be tested: sector 3-4, where the incident occurred, and the adjoining sectors, 2-3 and 4-5.
Looks like standard splicing. And, yes, platinum & platinum alloy used in high temp thermocouples.
So no operations this year after all. I’m willing to wager that the other sectors will need to be warmed up. So all these articles talking about how the LHC will valiantly run through the winter are now moot. I see from the article that Peter is not convinced that the LHC will ever run at 7 tev. This project is deeply disapointing both qulitatively and quantitatively.
It still seems possible they will not have to warm up the other sectors, looks like we’ll find out within the next couple weeks.
From what I’ve read, getting all the magnets to reliably work at high
enough fields to do 7 TeV may never be possible, but maybe something like 6.5 TeV is, and the difference of .5 TeV is not of big significance.
I suspect that the LHC will ultimately be a significant scientific success, although it may be a public relations failure. Promoting the thing as possibly discovering extra dimensions, and making a big public fuss over it while it is still being tested and debugged (something that always takes longer than expected…) seems to me to have been a mistake.
I know this may not help, but it took over two decades for the 200-inch Hale Telescope on Mount Palomar to be completed, and it turned out to be a pretty useful instrument for astronomy.
And then there is the Hubble….
Would you at least agree that the design seems unsound. If you must spend months warming and cooling every time there is a bad wire you are looking at years of stoppages for every little flaw immaginable . . . like a bad wire. God forbid you get something major like the explosion that was not an explosion last year. If this were an airplane, many people would be dead by now.
With regards to my latest comment above. I know because it has been pointed out that even wires at the LHC are special and made up of gold and platinum and angel dust. But I cannot imagine a smaller increment of tecnology than a wire. Except perhaps a bolt. But I bet even bolts there are fashioned by the elves from precious metals.
It’s not at all clear that the design is unsound. So far the fact that it takes time to warm up and cool down sectors has not been the main thing delaying them (rather it has been things like bad magnet support design, bad PIMS, bad splices). Their problems have been pretty much the kind of thing you would expect when you first try and run an extremely complicated piece of equipment that has never operated before.
It’s not an airplane, it’s a tricky piece of scientific apparatus, and its problems have put no one in physical danger. When the thing is running, people are kept far away, and it should be able to explode perfectly safely, if that’s what happens.