-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathmissing-images.html
More file actions
153 lines (123 loc) · 10.5 KB
/
missing-images.html
File metadata and controls
153 lines (123 loc) · 10.5 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
<!DOCTYPE html>
<html lang="en" data-content_root="./">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Processing Sequences with Missing Images — DIALS documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
<link rel="stylesheet" type="text/css" href="_static/basic.css?v=b08954a9" />
<link rel="stylesheet" type="text/css" href="_static/alabaster.css?v=27fed22d" />
<link rel="stylesheet" type="text/css" href="_static/sphinx-design.min.css?v=95c83b7e" />
<link rel="stylesheet" href="_static/dials-styles.css" type="text/css" />
<script src="_static/documentation_options.js?v=5929fcd5"></script>
<script src="_static/doctools.js?v=fd6eb6e6"></script>
<script src="_static/sphinx_highlight.js?v=6ffebe34"></script>
<script src="_static/design-tabs.js?v=f930bc37"></script>
<link rel="icon" href="_static/favicon.ico"/>
<link rel="author" title="About these documents" href="about.html" />
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="User guide for scaling data with DIALS" href="dials_scale_user_guide.html" />
<link rel="prev" title="DIALS How-To guides" href="howto.html" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
</head><body>
<div class="logoheader container">
<a href="index.html">
<img class="logoheader" alt="DIALS" src="_static/dials_header.png" />
</a>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="processing-sequences-with-missing-images">
<h1>Processing Sequences with Missing Images<a class="headerlink" href="#processing-sequences-with-missing-images" title="Link to this heading">¶</a></h1>
<p>DIALS treats sequences as a contiguous set of rotation images, and in some circumstances deviations from this will cause problems. A clear example of this is having one or more “bad” images in your data set - simply removing them <em>will not</em> be enough to allow processing to complete smoothly.</p>
<section id="importing">
<h2>Importing<a class="headerlink" href="#importing" title="Link to this heading">¶</a></h2>
<p>Importing the images with <code class="docutils literal notranslate"><span class="pre">template=blah_####.cbf</span></code> will not work, as this checks for continuous images, and it will be necessary to use <code class="docutils literal notranslate"><span class="pre">allow_multiple_sequences=true</span></code>. After this, finding spots and indexing work as usual, but with multiple imagesets.</p>
</section>
<section id="refinement">
<h2>Refinement<a class="headerlink" href="#refinement" title="Link to this heading">¶</a></h2>
<p>Refinement <em>without</em> <code class="docutils literal notranslate"><span class="pre">scan_varying=true</span></code> will work fine, but following that, scan varying refinement will fail with an error:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Sorry</span><span class="p">:</span> <span class="n">A</span> <span class="n">single</span> <span class="n">scan</span><span class="o">-</span><span class="n">varying</span> <span class="n">crystal</span> <span class="n">model</span> <span class="n">cannot</span> <span class="n">be</span> <span class="n">refined</span> <span class="n">when</span> <span class="n">associated</span> <span class="k">with</span> <span class="n">more</span> <span class="n">than</span> <span class="n">one</span> <span class="n">scan</span> <span class="ow">or</span> <span class="n">goniometer</span>
</pre></div>
</div>
<p>The issue here is that scan-varying refinement requires that each crystal being refined is associated with a single scan. In our case, we have a single crystal model from indexing, but this is associated with multiple scans. To proceed, we can split the experiment list into individual files. This breaks the sharing of models between experiments by making copies of each model for each file:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dials</span><span class="o">.</span><span class="n">split_experiments</span> <span class="n">indexed</span><span class="o">.</span><span class="n">refl</span> <span class="n">indexed</span><span class="o">.</span><span class="n">expt</span>
</pre></div>
</div>
<p>From this point, we could process each block as per the usual tutorial instructions (ideally in separate directories). However, this will refine the beam and the detector, which <em>should</em> be shared, separately for each process. A better way to proceed would be to recombine the experiments as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dials</span><span class="o">.</span><span class="n">combine_experiments</span> <span class="n">experiments_</span><span class="o">*.</span><span class="n">expt</span> <span class="n">reflections_</span><span class="o">*.</span><span class="n">refl</span> \
<span class="n">reference_from_experiment</span><span class="o">.</span><span class="n">goniometer</span><span class="o">=</span><span class="mi">0</span> \
<span class="n">reference_from_experiment</span><span class="o">.</span><span class="n">detector</span><span class="o">=</span><span class="mi">0</span> \
<span class="n">reference_from_experiment</span><span class="o">.</span><span class="n">beam</span><span class="o">=</span><span class="mi">0</span>
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">reference_from_experiment</span></code> options tells <code class="docutils literal notranslate"><span class="pre">dials.combine_experiments</span></code> to take the goniometer, detector and beam models only from the first experiment (although any would have done as these are merely copies of each other). The resulting <code class="docutils literal notranslate"><span class="pre">combined.expt</span></code> file has a separate (but identical) crystal model for each scan, alongside shared goniometer, detector and beam models. Scan-varying refinement and integration can now proceed as usual:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">dials</span><span class="o">.</span><span class="n">refine</span> <span class="n">combined</span><span class="o">.</span><span class="n">expt</span> <span class="n">combined</span><span class="o">.</span><span class="n">refl</span> <span class="n">scan_varying</span><span class="o">=</span><span class="n">true</span>
<span class="n">dials</span><span class="o">.</span><span class="n">integrate</span> <span class="n">refined</span><span class="o">.</span><span class="n">expt</span> <span class="n">refined</span><span class="o">.</span><span class="n">refl</span>
</pre></div>
</div>
</section>
<section id="export">
<h2>Export<a class="headerlink" href="#export" title="Link to this heading">¶</a></h2>
<p>Currently, <code class="docutils literal notranslate"><span class="pre">dials.export</span></code> will not allow MTZ export of the multiple-experiment integration, but we can rely on <code class="docutils literal notranslate"><span class="pre">dials.split_experiments</span></code> again:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>dials.split_experiments integrated.expt integrated.refl
i=0
for e in $(ls experiments_*.expt)
do
r=reflections_$i.refl
echo "dials.export $e $r mtz.hklout=integrated_$i.mtz"
((i++))
done
</pre></div>
</div>
</section>
</section>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="Main">
<div class="sphinxsidebarwrapper"><!--<h3>Navigation</h3>-->
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="index.html">Home</a></li>
<li class="toctree-l1"><a class="reference internal" href="about.html">About</a></li>
<li class="toctree-l1"><a class="reference internal" href="installation.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="documentation/index.html">Documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="documentation/tutorials/index.html">Tutorials</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="howto.html">How-to</a></li>
<li class="toctree-l1"><a class="reference external" href="https://dials.github.io/kb">Knowledge Base</a></li>
<li class="toctree-l1"><a class="reference internal" href="workshops/index.html">Workshops</a></li>
<li class="toctree-l1"><a class="reference internal" href="national_resource.html">US National Resource</a></li>
<li class="toctree-l1"><a class="reference internal" href="publications.html">Publications</a></li>
<li class="toctree-l1"><a class="reference internal" href="metrics.html">Metrics</a></li>
<li class="toctree-l1"><a class="reference internal" href="projects.html">Projects</a></li>
<li class="toctree-l1"><a class="reference internal" href="license.html">License</a></li>
</ul>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer container">
<a href="https://www.diamond.ac.uk/"><img class="logofooter" alt="Diamond" src="_static/diamond_logo.png" /></a>
<a href="https://www.ccp4.ac.uk/"><img class="logofooter" alt="CCP4" src="_static/CCP4-logo-plain.png" /></a>
<a href="https://www.stfc.ac.uk/"><img class="logofooter" alt="STFC" src="https://dials.github.io/images/logos/ukri-stfc.png" /></a>
<a href="https://www.lbl.gov/"><img class="logofooter" alt="LBL" src="https://dials.github.io/images/logos/lbl.png" /></a>
<a href="http://smb.slac.stanford.edu/"><img class="logofooter" alt="SSRL/SMB" src="https://dials.github.io/images/logos/smbssrl.jpg" /></a>
</div>
<script type="text/javascript">
$(document).ready(function() {
$(".toggle > *").hide();
$(".toggle .header").show();
$(".toggle .header").click(function() {
$(this).parent().children().not(".header").toggle(400);
$(this).parent().children(".header").toggleClass("open");
})
});
</script>
<div class="footer">
©2026, Diamond Light Source, Lawrence Berkeley National Laboratory and STFC.
</div>
</body>
</html>