Instrument settings - incident_angle allowable formats

[jfkcooper]

I am just writing a .ort file saving algorithm for Mantis and realised that the current standard doesn't allow for (or isn't worded so it is clear it allows) a list of angles which were measured at.
At Isis it is common to measure at several angles and then stitch the datasets together (the correctness of doing this is a discussion for another time). The resulting stitched dataset will therefore be the product of several different runs taken at different angles. I wish to include these angles in the reduced data file, but as far as I can see, the allowable values are Union[value, vale range].
I propose to allow this to be Union[value, list[values], value range] instead so that a dataset which was measured at e.g. 0.3, 0.7, and 2.0 degrees can include these explicitly.
I would also say Wavelength could be similarly modified such that the wavelength ranges used at each of the angles could be included, making it Union[value, list[values], value range, list[value range]].
Any thoughts on this are appreciated

[arm61]

This seems like something that would have been thought about by @jochenstahn, @aglavic, or @bmaranville?

[jochenstahn]

This is a topic we discussed before - but there was no decision made.....
Please have a look at
https://www.reflectometry.org/file_format/specs_discussion
at the section 'stitched data'. Please comment on the suggestions there and probably add your own.

[andyfaff]

Do we allow for having multiple incident_angle entries?

#    measurement: 
#         instrument_settings:  
#             incident_angle:  
#                 magnitude:  0.65
#                 unit:       deg    
#                 error:      0.01
#             incident_angle:  
#                 magnitude:  3.0
#                 unit:       deg    
#                 error:      0.01

[jfkcooper]

How would/could this be input?

[jochenstahn]

I am not sure about multiple entries of the kind andyfaff suggested. Maybe Artur and Brian can comment on this?

The second suggestion in the specs_discussion page, i.e.

#        data_files:
#           - file: ....
#             timestamp: ....
#             details: 1.0 deg, 42 min

has the advantage that the angle (and other information) is assigned to an input file. One could even go to

#        data_files:
#           - file: ....
#             timestamp: ...
#             incident_angle: ...
#             wavelength: ...

This can always be done since (to my understanding) the key words incident_angle and so on are not looked up here.

Am I right that this additional information won't be used in the further data processing, but it rather helps a potential reader to understand what has been done?

[bmaranville]

I believe there's a way to enter multiple datasets with different header information for each - this is how polarized composite datafiles work, for instance. You could put a different value of datasource.measurement.instrument_settings.incident_angle for each dataset, if you wanted to keep them separate after reduction, or you could make a column that has a value of the incident angle for each Q and R reported (we still have to all agree on how that will be uniquely identified as a column, see #70)

[jfkcooper]

My approach is currently somewhat pragmatic, as I don't really want the saving into an .ort format to require a rework of the reduction schema, so what I have is a single workspace entity which has the stitched data, and most of the information about what went into it. My thoughts are that something like:

#    measurement: 
#         instrument_settings:  
#             incident_angle:  
#                 individual_values: 0.3, 0.7, 2.0
#                 unit:       deg 

As was previously mentioned would be a potential unambiguous solution

*edited after finding the relevant part of previous discussion - sorry

[bmaranville]

We don't have that sort of entry in our schema, right now - how would you figure out which Q points correspond to which incident_angle values, if you have a long list of Q?

[jfkcooper]

With stitched data you can't I'm afraid, though by also changing the Wavelength to be a list of ranges then it would imply the wavelengths/angles

[jfkcooper]

As I mentioned, the correctness of stitching data is another matter, but at least at ISIS the angular resolution matching and extremely high wavelength resolution means that it isn't a big practical issue to stitch.

[bmaranville]

How would you use the wavelength list + angle list to get the wavelength and angle for a given Q? I'd like to hear about how you work with stitched data (no criticism implied by all these questions - just curiosity)

[jfkcooper]

If we had:

#    measurement: 
#         instrument_settings:  
#             incident_angle:  
#                 individual_values: 0.3, 0.7, 2.0
#                 unit:       deg 
#         wavelength:
#                    individual_values: [2.0,14.0], [2.0,14.0], [1.5,14.0]

This makes the accessible Q points for the three angles
[0.0047,0.033], [0.011,0.077], [0.031, 0.29]
which do overlap, but give the majority contributor at a given Q. Not ideal tbh, but I thought including the information would be better than not.

[andyfaff]

If you're limited to two or three angles stitching isn't a problem.
It would be good to store those two or three angles in the header. It s not really any different in character to storing an angular range. I typically go back to old files to figure out how we measured the data, when we're setting up a new experiment.

We typically work with the equivalent of a QProbe and you don't need to know what the angles or wavelengths are, just q and dq are sufficient.
The only time you need wavelength and angle is if you have some sort of absorption to take care of. If that's necessary you can just use extra columns.

[andyfaff]

@jochenstahn said:

Am I right that this additional information won't be used in the further data processing, but it rather helps a potential reader to understand what has been done?

This is mostly correct, in that we mostly use it to go back and figure out what angles we used.

However, I still think they need to be machine readable/retrievable.

It's not necessarily suitable to store them as a column.

• the nominal angle of incidence is not necessarily the same as the actual angle of incidence for a given Q-point, due to gravity. So now you have to have two columns, to store both.

• a given Q point may not be associated with a given angle of incidence. If a detector has high resolution and wavelength binning is fine, then individual detector pixels may be rebinned in Q. For a given detector image each individual detector pixel may have the same nominal angle of incidence, but they all have unique actual angles of incidence. This uniqueness arises from gravity effects (different wavelengths have different angles of incidence), but also from the divergence of the beam - at a given wavelength one side of the specular ridge will have a lower angle of incidence than the other side of the specular ridge. The rebinning process lumps together pixels of similar Q, but I don't think it's possible to extract an actual angle of incidence associated with each bin. In such a case you probably want to store the three angles in the header.

My thought is we want the ability to do both. In the header we'd like to store individual incident_angle (probably 2/3), but also retain the ability to have various angle columns.