SCORS- Correlator Application User Guide
The Correlator application facilitates stratigraphic correlation of cores from multiple holes at a drill site Its features support depth shifting of cores based on high-resolution core logging data and the affine constraint to construct a core composite depth below sea floor (CCSF) depth scale, and splicing of selected core intervals to construct the most complete stratigraphy possible at a site.
On the JOIDES Resolution (JR), the Correlator application is used as part of the stratigraphic correlation support (SCORS) ecosystem, in conjunction with additional software applications covered in a separate user guide (Table 1-21, Fig. 1-21).
Figure 1-21: Overview of SCORS tools and data flow and how they relate to the Correlator application. The yellow number labels 1 through 4 represent the sequence of use of applications when using Correlator as the main tool.
Use of the Splice File Fixer app is recommended when using an alternate tool for correlation.
Software application | Task | Documentation |
Correlator | Stratigraphic correlation of cores from multiple holes at a drill site, by depth shifting cores using high-resolution core logging data and constructing a core composite depth below sea floor (CCSF) depth scale, and by splicing selected core intervals to construct the most complete stratigraphy possible at a site. | scors_correlator_ug_20190321 |
Correlation Downloader | Download core logging data from the LIMS database, with options to filter the data as appropriate, and to append new core data to hole files. | scors_lims_ug_20190321.docx |
Splice File Fixer | Check if the splice intervals generated by an external application are consistent with the core and section information in the LIMS registry before upload. This task is not needed anymore if Correlator v. 2 or later is used because it already uses the necessary section information from LIMS. | |
SCORS Uploader and Manager | Load affine tables and splice interval tables created by an external application (e.g., Correlator) to the LIMS database where the information can be reported and applied to all data in LIMS. | |
LORE Reports for Stratigraphic Correlation | Provide (A) lists of existing affine tables and splice interval tables, with links to uploaded user files; (B) detailed, LIMS-computed affine and splice interval tables; (C) CCSF (alternate) depths for any data set in the LIMS; and (D) data sets by selected splice. |
This user guide is for Correlator Version 3.0 released in March 2019. Find the application on the following JRSO web sites:
Follow your browser and operating system’s instructions to install and launch the application.
Access to the applications on the ship will be obvious, or assisted by JRSO staff, when you board the JR.
Correlator installs a directory for its internal database at a default location that looks something like this:
Documents [or other system folder]
Correlator
3.0 [or similar version number]
db
default.cfg
log
temp
When users import correlation data from a directory of choice, Correlator indexes the data and places them in this directory for internal use. Users doesn’t have to be aware of this directory but can change the path if needed (Fig. 1-31):
We don’t expect a significant need to import legacy affine tables. Therefore, the following feature has been minimally tested. Ignore this section if you create your affine tables with version 3.
Affine tables created in Correlator version 3 differ from those in version 2: they include additional information (last three columns, see Appendix 2) that allow Correlator to reconstruct the relationship between cores. The following details will become clear once the user is familiar with version 3.
Affine tables created in the last production version, 2.1_rc2, can still be imported in 3.0 by right-clicking "Saved Tables" in the Data Manager (available once data are loaded) and choosing Import legacy affine table. However, because legacy affine tables have no record of the reference core used to create a TIE, all cores shifted by a TIE will be converted to status REL on import into 3.0. This has no effect on their cumulative offsets, which will be preserved, but users will have to rebuild the TIEs manually (as was the case in version 2).
Correlator operates within a single window that can be expanded as much as available monitor space permits. The window can be toggled between the Data Manager view (Fig. 1-41) and the Display view (Fig 1-42). To navigate between these views, use the top button on the floating tool bar (Figs. 1-41 and 1-42), which is always available on top of either view unless turned off in the View menu (Fig. 1-43).
The Data Manager view has two functional tabs across the top:
The Display view has three areas, two for data plotting and one for the functional controls, organized in four control tabs.
The Display Preferences are self-explanatory and not further described here. Additional, more fundamental display preferences are available in the View menu (Fig. 1-43).
The Data Filters controls are largely self-explanatory but require some commentary (Fig. 1-44).
Figure 1-44. Three data filtering options on the Data Filter tab: Decimate, Gaussian Smoothing, and Cull. Shown is a 9-point Gaussian filter applied to the magnetic susceptibility data whereby both original and smoothed (white trace overlay) are plotted. The filter can be edited or deleted.
Before you can start importing correlation data files, you need to store them in a compatible format in a directory of choice. Content of each data file must be for one hole and one data type. This is automatically true if downloading LIMS data using the Correlation Downloader application, which is highly recommended when working in conjunction with the LIMS infrastructure, such as on the JR, as this offers numerous other advantages.
It is best practice to organize the data in site folders.
When you first start up Correlator, you’ll find the empty Data Manager view with only one item, labeled:
To import data files:
Correlator normally detects the commonly used data types upon import, based on the header label of the data column. However, here is what you do if that should not be the case and you are getting an error message upon trying to import.
Once you have imported, loaded and displayed you first data set, and you then go back to the data manager to Add new data, you will get the error message in Fig. 2-14. Click OK. This is simply to make you aware of how Correlator works. After you have imported the new data, you need to Load the data again anyway to get a current display.
Correlator needs top and bottom depths of sections to compute splice intervals correctly. The application maintains an internal table for that purpose. Three options exist due to the evolution of the program and its use in different environments.
In Correlator version 1, the tops and bottoms of sections were exclusively computed by Correlator based on the imported data, which often resulted in small and sometimes larger errors in splice tables. This functionality still exists in version 3 for backward compatibility, however, it is disabled by default in the new Correlator/Preferences menu (Fig. 2-21). In situations where users do not have a section summary file for upload, they can enable the Infer Section Summary if none is provided feature and for each data file loaded (those with "Enable" on in the Data List), Correlator gathers the minimum and maximum depths of data points for each section. If multiple data types are loaded for a hole, all data types are considered and the minimum of minima offsets is used as the top of section and the maximum of maxima offsets is used for the bottom of section.
If enabled, Correlator will give a message at the time of data import that the section summary was inferred from the data (Fig. 2-22).
Starting with Correlator version 2, a section summary file is uploaded automatically when the user imports data to Correlator, if that section file exists in the same directory as the data file(s). This is the standard process on the JR because the Correlation Downloader application automatically downloads a section summary file from LIMS along with the data files, and also updates the section file along with the data file when a new core is appended in the real-time correlation workflow. The user will see the section summary file in the data folder but must not upload it separately (Correlator would try to load it like a regular data file). If the section summary does not exist or cannot be uploaded because of an incompatible format, an error dialog appears.
Alternatively, section summary files can also be imported “manually” by right-clicking on the Section Summaries item in the Correlator Data Manager and selecting the Import Section Summary Files(s) option. Once the user browses to the appropriate file and selects it, an import dialog window opens showing (part of) the file content (Fig. 2-23). The user can click Import or Cancel.
Note: the section summary can also be uploaded the same way as any data file, but that is usually meaningless because we don’t need to plot the sections that way. If you need to upload section summaries manually, make sure you do so from the Section Summaries item so Correlator uses them as intended.
Once correlation data have been imported to Correlator and are listed in the hierarchy of the Data Manager, they reside in Correlator’s own local database. Users still need to load the data for display and correlation. To load data, select, then right-click on a line item in the Data Manager, and select Load.
If you import data and switch to the Display view without loading, a message will appear in the plot window reminding you that you need to load the data for them to be plotted.
During the typical JR work flow, correlation specialists download the correlation data from LIMS each time the data from a new core are available. This iterative process is facilitated in at least two ways:
To update the Correlator internal database with updated files:
Updating the Correlator database is equivalent to an import. If you update data and switch to the Display view without loading, a message will appear in the plot window reminding you that you need to load the data for them to be plotted. Make sure you also load the affine and splice tables again, if applicable.
When working with the LIMS database, users on the JR will typically only be exporting affine tables and splice interval tables. Users will subsequently upload these two tables to the LIMS database for each correlated site, using the SCORS Uploader and Manager application, so all necessary data reports can be computed in, and reported from LIMS via the LORE reporting interface. This has the advantage that any LORE user can download any LIMS data type with the CCSF depth scale, or by splice, including data not used in the correlation process.
To export an affine or splice table, right-click on the item and select Export from the context menu. Browse to your data folder for the site (if necessary) and save.
When you export affine and splice tables, you give them a name, and Correlator adds extensions *. affine.csv and *.sit.csv, respectively. It is advisable that you assign names to these files that make sense as the files will ultimately be uploaded to, and available from the LIMS database. In particular, use a naming scheme that helps you and others recognize associated affine and splice tables.
Should you import an affine or splice table that you exported in an earlier session, be aware that Correlator will rename the files according to its internal rules.
The correlation data imported to, and managed in Correlator can also be exported using the context menu for data items in the Data Manager. Such exports would make sense if users worked outside of the LIMS ecosystem and were able to apply the enabled affine and splice tables. This is currently not possible and exporting correlation data has no use case.
Seven different data management functions are accessible via context menus on the Data Manager tab (Table 2-61).
When opening Correlator for the first time, only one item is present in the Data Manager tab, called Root and it has only one function: Add new data. As soon as the first data file is imported, Correlator creates a data directory for the site, using the site name from the data file, and the following data group folders with all the data management functions appear:
Function (Menu item) | Function description | Root level | Site level | Data group items | Data items |
Add new data | Import data from directory | YES | YES | YES | |
Load | Load data for plotting in Display view. | YES | YES | YES | |
Update | The specified file is re-imported and the information in Correlator is updated. | YES | YES | YES | |
Delete | Remove from the Correlator database. | YES | YES | YES | |
Disable/Enable | Disable or Enable (should read “Enabled” and “Disabled”) status determines whether the data are loaded and can be plotted in the Display view. | YES | YES | ||
Export | Mainly used to generate affine and splice tables as CSV files that can be uploaded to LIMS. | YES | YES | ||
View | Brings up a modal window with the data in a grid. | YES |
The goal of depth shifting is to construct a composite depth below seafloor (CCSF) depth scale by arranging the cores from one or more holes in a way that the combined data better represent the stratigraphy at a site than the cores do at their standard CSF-A depth scales.
The composite depth scale is defined in an affine table where each core has a cumulative offset, the difference between its CCSF and CSF-A depths. The affine constraint stipulates that no virtual stretching or squeezing is allowed within a core. This is a practical and effective approach because (A) it is based on straightforward rules and avoids the implementation complexity of virtual stretching and squeezing, and (B) sampling of the physical cores would become overly complicated if the CCSF depth scale was distorted in the correlation process relative to the CSF-A scale, which is true to the physical samples. Correlation under the affine constraint creates a “~95%” solution quickly and effectively, whereas resolving the remaining “~5%” of the correlation involves a lot more work and subjectivity and the result is far more difficult to apply.
The coring process does stretch and squeeze cores to a certain degree, and this artificial stratigraphic distortion, combined with actual variability in the stratigraphy in multiple holes, means that between two cores only one stratigraphic feature can be exactly correlated and assigned the same composite depth. Other correlative features will have somewhat different composite depths.
Depth shifts can be defined by one of two methods: by creating a tie between two cores (TIE method) or by shifting a fixed amount or percentage of depth (SET method). A core shifted with the SET method can later be shifted by the TIE method, however, cores that are tied (part of a chain) cannot be shifted by the SET method. Cores have a type designation based on the (last) shift method applied: REL, TIE, or SET. These designations are used programmatically to implement the above rules and to control the color of the core data trace.
Correlator keeps track of the cumulative offset of each core resulting from all shifts applied, the method used for the latest shift, and in the case of TIE relationships, the core to which a given core was tied. The specification for the affine table was expanded with Correlator version 3 for the purpose of recording the latest shift method and the tie relationships (see Appendix 2 for the affine table specification).
Coring in one hole can never recover a complete stratigraphic sequence. Ideally, two holes can achieve that if the cores in the second hole are offset sufficiently to cover the coring gaps in the first hole. A CCSF scale is then defined by tying cores from the top (as close to seafloor as possible) to the bottom (as deep as correlations are possible), establishing ties from reference (REF) cores to shifting (SHIFT) cores, thus creating a simple chain of cores.
In many if not most cases, cores from two holes cannot achieve complete stratigraphic coverage due to a number of factors and cores from a third or even fourth hole are needed. Since the goal is (should be) to tie all cores from all holes into the CCSF framework, the cores will not form a simple chain anymore, but include multiple chain branches. A new chain branch is created when a core serves as a REF core for two or more SHIFT cores. Under no circumstance can a core be a SHIFT core to more than one REF core due to the affine constraint.
Because the CCSF scale construction proceeds from top to bottom, “upstream” tie changes, i.e., changing the tie from a SHIFT core to its REF core, are simply forbidden. Upstream tie changes would have undesirable and unintended effects. In such cases, the user needs to decide where to implement the tie change further upstream so the tie break effect is directed downstream. Downstream directed tie changes preserve the ties unless a chain branch must be broken off, which the program can identify and the user can be prompted to accept the consequence or cancel the action. An example will be given below.
The user can shift cores by establishing tie points for two cores on the basis of correlative features in the data or by opting for a statistical correlation.
To shift cores, select the Shift Cores tab of the Display view to see the control pane (Fig. 3-21), which provides both the controls for shifting cores and the feedback on the progress of shifting.
At the top of the Shift Cores control pane three plots are available to choose from:
Beneath the graphic are the controls for depth shifting. The most important tie shift controls are also available as context menus.
Core traces are colored based on the depth scale and shift status of the cores. At the beginning, all core traces are yellow, indicating that they were not shifted and are still at the original CSF-A depth scale (Fig. 3-22). They have the status REL, meaning their positions are relative to the previous core as defined by the CSF-A scale. The fact that yellow traces and status REL go together is trivial at this stage but not later on.
To prepare the first tie, we select the core with the shallowest data as the “anchor” or “root” core as the reference (REF) core. In the case of our example (Fig. 3-23), this is core B1:
A. B.
Figure 3-25. The user can pick one of two scope options: “This core and all related cores below” or “This core only”. Shown are (A) the options on the context menu and (B) the options on the control pane.
The remaining controls on the Shift Cores tab (SET, Undo and Save) are described in subsequent sections.
What happens when you change your mind about a tie between two cores and want to redo it? Or what if you created a chain using cores from the first two holes, and now core data arrive from the third hole and it makes sense to “weave them in”? The latter scenario is exemplified by Fig. 3-26 where it is obvious that the first core gaps in the first two holes align exactly, meaning that correlation of the first two cores to subsequent cores is not possible using only Holes A and B. We are therefore introducing additional data from a third Hole D.
Ties can be revised anywhere in an existing chain as long as the change is made downstream, i.e., in the direction of the existing tie, or if a new core (e.g., from Hole D) is added as a SHIFT core anywhere in the chain. All ties from related cores below the SHIFT core remain intact if the user selects to shift “all related cores below”.
Two exceptions exist.
It may happen that you want to (or have to) break out a core from the chain because you found better chaining options. The easiest thing may be to delete a tie or two and correlate them again using suitable correlative features. This is now easy:
Alternatively, you can break a tie via the control pane:
It is then your task to fix whatever consequences your action has. Correlator assists you by coloring the broken-out cores orange, representative of the status type REL.
Sometimes you may want to shift one or more cores by a certain distance or a certain percentage, rather than by tying them. Example use cases are:
To set one or more cores:
The following happens when you apply a shift with the SET method:
Note: With the first two of the three SET scope options, cores that are in a TIE relationship cannot be SET. They first need to be un-tied (Fig. 3-33).
Note: The percentage or absolute shifts are always relative to the original CSF-A depth, not relative to the depth cores may already have been shifted! For example, a core may already have a cumulative offset of 5 m and if an absolute shift of 1 m is applied the core actually shifts 4 m upward, not 1 m downward.
You can undo shifts in two principal ways: one shift at a time, or all shifts in one swoop.
To undo shifts one at the time:
Undo all shifts in one action by disabling the affine table:
Undo shifts in one action by deleting the affine table:
Depth shifts are saved in an affine table within the Correlator application. Users can save as many affine tables as they like. Users have three ways to trigger a save of recent shift changes to the affine table, each:
If no enabled affine table exists, the current shifts will be saved in a new, enabled affine table without any prompt or confirmation. The file name generated by Correlator will include a number that is incremented by 1 from the highest numbered affine in Correlator, including disabled affines.
If an enabled affine table exists, you have the option of updating an affine rather than creating a new one.
Upon saving, you will receive a confirmation (Fig. 3-53).
If you want to use an affine table that already exist in Correlator but is disabled:
To export your final affine table (i.e., to load it into the LIMS database), use the Export option on the context menu (Fig. 3-54).
In some cases, you may want to import an affine table that you exported in a previous session. To do so, right-click on the Saved Tables item in the Data Manager and select Import affine table. (Fig. 3-55).
Note the option to Import legacy affine table, which allows the import of affine tables created prior to Correlator v. 3.0.
A splice is constructed by selecting cores from multiple holes that were stratigraphically aligned relative to a common core composite depth below seafloor (CCSF). The specification of the splice interval boundaries, where a core interval from one hole is spliced to a core interval from another hole, is a subjective matter. Ideally the splice interval boundaries are at positions where high-resolution stratigraphic features are at the same CCSF depth. This is the reason why the choice of splice interval boundaries should be on the correlation specialist’s mind when selecting TIE points in the depth shifting process.
Basic rules:
Before starting to splice, go to the Preferences control panel and make sure that
Then switch to the Splice Interval tab to show the control panel for splicing on the right side of the plot window (Fig. 4-21).
Next, repeat the following for each core you want to include in the splice:
NOTE: Correlator doesn’t care from which data type you drag a trace, it simply uses whatever trace you drag over. It normally makes sense to drag traces from only the same data type. In the end, any data type can be retrieved by splice.
You may have created a tentative splice using cores from holes A and B, and now that depth-shifted cores are available from hole C you would like to splice some of them in, or you simple decided on better interval from another core. If you try to drag a core over an existing splice, Correlator simply won’t add it because no more than two overlapping core intervals can be in any one splice depth interval. You first need to create a gap, which requires you to split the splice.
To create a gap and insert a new core interval:
Ideally, depth shifting should be concluded before a splice is assembled. However, life is not ideal and you may want to shift a core after you have made a splice. Because splice intervals are defined by their CCSF depths, shifting their cores invalidates the splice interval boundaries.
If you are trying to shift a core that is part of the splice, Correlator will warn you that this (and other related cores, if so opted) will be removed from the splice (Fig. 4-41):
NOTE: If you commit to the shift and then regret it, the shift can be undone with the Undo Previous Shift button. The shift ties are re-established, however, the splice intervals that were removed are not replaced. Here is a simple workaround to get the splice back:
If the user has at some point created an alternate splice, the cores with compromised splice interval boundaries are not removed from that splice. Only if and when the user switches to that alternate splice using the Alternate Splice button will they find a surprise in the form of an error like this:
This is because Correlator keeps track of the affine offsets in the splice registry and compares the offsets from the affine and the splice when the user is loading the tables. It allows loading only if no difference is detected.
The basic functions and dialogs to save, enable and disable, load, export and import, or delete splice tables works in principal the same way as for affine tables, described in the section Manage affine tables.
Two important exceptions exist for loading splices:
If you export affine and splice tables, you give them a name, and Correlator adds extensions *. affine.csv and *.sit.csv, respectively. It is advisable that you assign names to these files that make sense as the files will ultimately be uploaded to, and available from the LIMS database. In particular, use a naming scheme that helps you and others recognize associated affine and splice tables.
When you import an affine or splice table, be aware that Correlator will rename the files according to its internal rules.
On the Splice Cores tab, you’ll find a button Select Alternate Splice. This feature allows you to view another, existing splice in the third (right-most) splice plotting track. You will probably have to expand your splice window width to see that column.
When you click the button, you get the dialog in Fig. 4-53. You can specify a data type and a splice.
Alternatively, you can Go to Data Manager, open the Saved Tables tree, enable another splice and its associated affine table, and reload the tables.
Correlator benefits significantly from having a section summary file imported along with the correlation data. The table below shows the section summary exported automatically from LIMS when using the Correlation Downloader application. As long as the section summary file is located in the same folder as the correlation data, Correlator will import it automatically.
Current column header from LIMS & in Correlator v3 | Format | Example values | Correlator upload file requirements |
Exp | text | 361, 362T | required |
Site | 69, 987, 1103, U1423 | ||
Hole | A, B, C, ..... | ||
Core | integer | 3, 45, 138 | |
CoreType | text | H, X, R, .... | |
Section | 1, 2, ….12, CC | ||
Recovered length (m) | number | 0.1, 1.5 | optional |
Curated length (m) | |||
TopDepth | 2.3, 124.175, 1504.4 | required | |
BottomDepth |
The affine table exported from Correlator has the 15 columns described in the table below. In order to take advantage of the features offered from within the LIMS database, affine tables exported from Correlator or any other correlation programs must meet import requirements of the SCORS Uploader and Manager application. As a minimum requirement, the first 7 columns must be present.
Affine table column headers | Format | Examples values | LIMS upload file requirements | Upload validation | LIMS report sourcing |
Site | text | 69, 1103, U1423 | Presence of columns required | Validated against identities in LIMS | From upload file |
Hole | A, B, C, ..... | ||||
Core | integer | 3, 45, 138 | |||
Core type | text | H, X, R, .... | Not validated | From LIMS computations | |
Core top depth CSF-A (m) | number | 2.3, 124.175, 1504.4 | |||
Core top depth CCSF (m) | |||||
Cumulative offset (m) | -0.560, 4.562, 48.392 | Number format validated | From upload file | ||
Differential offset (m) | Presence of columns optional | Not validated | |||
Growth rate | 0.91, 1.05, 1.21 | ||||
Shift type | text | REL, TIE, SET | |||
Data used | NGR | ||||
Quality comment | “Interval includes core disturbance” | ||||
Reference core | A3, B15, C123 | Not reported from LIMS | |||
Reference tie point CSF-A (m) | number | 2.3, 124.175, 1504.4 | |||
Shift tie point CSF-A (m) |
The splice table exported from Correlator has the 15 columns described in the table below. In order to take advantage of the features offered from within the LIMS database, splice tables exported from Correlator or any other correlation programs must meet import requirements of the SCORS Uploader and Manager application.
Current header in Correlator 2 and 3 | Format | Examples | LIMS upload file requirements | LIMS upload validation | LIMS report sourcing |
Site | text | 69, 987, 1103, U1423 | Presence of all columns is required | Validated against identities in LIMS | From upload file |
Hole | A, B, C, ..... | ||||
Core | integer | 3, 45, 138 | |||
Core Type | text | H, X, R, .... | |||
Top Section | 1, 12, CC | ||||
Top Offset | number | 1.2, 45, 151.7 | |||
Top Depth CSF-A | 2.3, 124.175, 1504.4 | Not validated | From LIMS computations | ||
Top Depth CCSF-A | |||||
Bottom Section | text | 1, 12, CC | Validated against identities in LIMS | From upload file | |
Bottom Offset | number | 1.2, 45, 151.7 | |||
Bottom Depth CSF-A | 2.3, 124.175, 1504.4 | Not validated | From LIMS computations | ||
Bottom Depth CCSF-A | |||||
Splice Type | text | TIE, APPEND | From upload file | ||
Data Used | NGR | ||||
Comment | “Might include slumped bed” |
This user guide was written by Peter Blum, March 2019 for Correlator Version 3.0. See archived version for the user guide for the previous version of SCORS- Correlator.
Correlator User Guide - 27Sept2022
Stratigraphic Correlation Support (SCORS) User Guide.docx: Word file written by P. Blum (2016-10-8)