9
$\begingroup$

I am a software engineer and I am preparing a presentation to aspiring astronomy PhDs on how to use best-practice software engineering when publishing code (such as include documentation, modular design, include tests, ...). In particular my presentation will be focused on publishing "pipeline" code, that is, code that is mainly focused on extracting and transforming data from a particular dataset to a suitable shape for analysis.

I am trying to find good example of published astronomy pipelines that I can point students to, but as I am not a astronomer I am struggling to do so, so would like your help. It doesn't matter if the published pipeline is super-niche or not very popular so long as you think it is engineered well so that I can look it up.

Improve this question
$\endgroup$
1
  • 4
    $\begingroup$ I'm working on the 4MOST pipeline at the moment, but it is very much "in progress". $\endgroup$
    – ProfRob
    Commented Apr 4, 2023 at 17:59

2 Answers 2

Reset to default
8
$\begingroup$

I think one really nice example right now is the James Webb pipeline. I haven't dived into the internals yet but I think it does a good job of demonstrating the best-practices of software engineering that you mentioned.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ I just looked it up, and indeed it is an excellent example well-engineered pipelines. $\endgroup$
    – nsimplex
    Commented Apr 5, 2023 at 6:41
3
$\begingroup$

Not a necessarily overly authoritative answer because I don't know first hand, but beyond RoySart's JWST pipeline recommendation, another example of something really massive might be that of the Vera C. Rubin Observatory "...that will photograph the entire available sky every few nights."

The reason I think so is the massive, firehose-like amount of data it will generate. The high cadence of re-observation will make it particularly sensitive as a instrument.

It will be setting off all kinds of alarms

much more than Zwicky

and so those data pipelines will need to be carefully vetted and tested and regularly tweaked and fine-tuned, and the only way for that to work efficiently is if they are well understood by a sizable group of astronomical programmers (programming astronomers? astro-engineers?). And the only way that's going to happen is if the whole thing is very well engineered and documented!

So while it may be too big to serve as an easy example for you, there might be some explanations you can digest and use in your course.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .