1. About Correlator 4.0

The Correlator application facilitates stratigraphic correlation of cores from multiple holes at a drill site. Its features support for

• depth shifting of cores based on high-resolution core logging data, including images, to construct a core composite depth below sea floor (CCSF) depth scale; and
• splicing of selected core intervals to construct the most complete stratigraphic representation possible at a site. 

On the JOIDES Resolution (JR), the Correlator application documented here is used in conjunction with other applications within the stratigraphic correlation support (SCORS) ecosystem (Fig. 1-1), which are covered in a separate user guide (Table 1-1). 

Figure 1-1. Stratigraphic correlation support (SCORS) applications.

Table 1-1. Stratigraphic correlation support (SCORS) application documentation

Version 4.0 was released on 31 March 2022. It can be downloaded at

<GitHub link>

On the JR you don't need to download the application - access will be obvious, or assisted by JRSO staff.

Correlator installs a directory for its internal database at a default location that looks something like this:

• Documents [or other system folder] > Correlator > 4.0 > db | default.cfg | log | temp

When you import correlation data from a directory of choice, Correlator indexes the data and places them in this directory for internal use. You don’t have to be aware of this directory but can change the path if needed (Fig. 1-2).

Figure 1-2. Correlator File menu.

Affine tables created in Correlator versions 3 and 4 differ from those in version 2: they include additional information (last three columns, see Appendix 2) that allow Correlator to reconstruct the relationship between cores. Affine tables created in the production version, 2.1_rc2 can still be imported in 4.0 by right-clicking "Saved Tables" in the Data Manager (available once data are loaded) and choosing Import legacy affine table. However, because legacy affine tables have no record of the reference core used to create a TIE, all cores shifted by a TIE will be converted to status REL on import into 4.0. This has no effect on their cumulative offsets, which will be preserved, but users will have to rebuild the TIEs manually (as was the case in version 2). 

2. Correlator basic functions overview

2.1. Data Manager view

Correlator operates within a single window that can be expanded as much as available monitor space permits. The window can be toggled between the Data Manager view (Fig. 2.1-1) and the Display view (next section). To navigate between these views, use the button in the lower left corner of either window (Figs. 2.1-2).

Figure 2.1-1. Data Manager view. Also shown is the floating tool bar, where the top button can be used to toggle to the Display view.

 
The Data Manager view has two functional tabs across the top:

• The Data Manager tab (Fig. X) lists the files imported into Correlator and provides the functions for adding and managing the data in Correlator.
• The Generic Data tab displays data from files being imported when such an import is triggered in the Data Manager tab (otherwise it’s an empty grid). 

2.2. Display view

Correlator operates within a single window that has two major views: Data Manager view (previous section) and Display view (Fig 2.2-1). The views can be toggled with the button in the lower left corner of either window (Figs. 2.2-2).

Figure 2.2-1. Display view, with the Display Preferences tab open on the right.

Figure 2.2.-2. Button two switch to Data Manager view.

 
The Display view has three areas, two for data plotting and one for the functional controls, organized in four control tabs.

• The left plot area is for depth shifting, and the corresponding controls are in the Shift Cores tab.
• The right plot area is for the construction of splices, and the corresponding controls are in the Splice Cores tab. This plot area can be turned off at the top of the Display Preferences tab if more display space is preferred for depth shifting.
• The control area can be toggled among the following four tabs:
  • Close: This tab acts like a button and closes the control panel. The control panel can be opened again by clicking any of the other four tabs.
  • Shift Cores: This tab has controls for depth shifting, which are described in the Depth shift cores section.
  • Splice Cores: This tab has controls for splicing, which are described in the Construct the splice section.
  • Display Preferences: This tab has general data display controls, as shown in Fig. X.
  • Data Filters: This tab offers data filtering options, including decimate, smooth and cull (Fig. X).

2.2.1. Shift cores tab

This tab has functions for shifting cores, which are explained in the Depth shift cores section.

2.2.2. Splice cores tab

This tab has functions for shifting cores, which are explained in the Depth shift cores section.

2.2.3. Display Preferences tab

This tab offers numerous display options that applying to core shifting and splicing. These options should be self-explanatory. Load some data, shift some cores, make some splices, and the simply explore the options be clicking around - nothing will affect the data.

Figure 2.2-3. Display preferences.

2.2.4. Data Filters tab

The Data Filters controls are largely self-explanatory but require some commentary (Fig. 2.2-3).

• Data filters are applied to the data from all cores for a specified data type. For a more specific filtering of known intervals with severe core disturbance, etc., you have the option to apply a file with those intervals specified in the Correlator Downloader application, at the time of data download.
• At this time, data culling can only be specified for top of cores (to remove data from “exotic” material washed down from higher up in the hole). A future version will include culling from core bottoms (“exotic” material “sucked in” when the piston core was removed) and culling from section ends (“edge effect”).
• Some odd behavior has been observed when deleting a cull filter under certain circumstances, and this has not yet been repaired (no user pressure). An easy workaround is to re-load the data.

Figure 2.2-4. Three data filtering options on the Data Filter tab: Decimate, Gaussian Smoothing, and Cull. Shown is a 9-point Gaussian filter applied to the magnetic susceptibility data whereby both original and smoothed (white trace overlay) are plotted. The filter can be edited or deleted.

2.3. Application menus

The application menus are rarely needed and presented here for completeness.

2.3.1. File menu

Figure 2.3-1. File menu.

2.3.2. Edit menu

Figure 2.3-2. Edit menu.

2.3.3. View menu

The View menu has a few fundamental display preferences not available in the Display Preferences tab of the Display view (Fig. 2.3.3-1.)

Figure 2.3-3. View menu.

3. Manage data in Correlator

3.1. Prepare correlation data

Data directories

Correlator is relatively flexible in terms of data file content and formats. Its import functions allow you to specify the only columns it really requires: depth and data. However, having the data files in a consistent format makes the process simpler and reliable. The following are the recommendations should be considered to be best practices on the JR.

• Use the Correlator Downloader application to download data from the LIMS database. Refer to the separate user guide for that application.
• Create a folder for each site (example in Fig. 3.1-1).
• Download one file for each data type and hole into the site folder. Append data for each new core measured using the Append core feature in Correlator Downloader.
• Create an images folder in each site folder where you place all reduced section images for the hole.
  • Batch-download the cropped section half images for a hole into a separate folder.
  • Use the R script provided to crop and reduce the images, then move the reduced images to the images folder in the correlation data site folder.
  • A separate user guide describes how to use the R script.
• If you have existing affine and splice tables for the site, place them in the site folder as well.

Figure 3.1-1. Example of a site folder with correlation data.

Special case of specifying data types

One of the primary objectives of stratigraphic correlation is to correlate section data from a new hole rapidly to the data from one or more previous holes to check if coring gaps in previous holes are recovered in the current hole. If coring gaps are not covered adequately due to the change in water depth (mostly due to tides, but also other environmental and operations factors), you want to provide relative coring offset (drill down interval) instructions to the drillers in near-real time.

In order to achieve this objective, we implemented a special task multisensor logger (STMSL), also referred to as ‘fast track’, with the same type of magnetic susceptibility (MS) instrument as the one mounted on the primary whole-round multisensor logger (WRMSL). The two instruments are the same for all practical purposes, however, users have found that slight differences in the data may exist (normally negligible compared to the overall analytical error). In any case, the ultimate goal is to measure all sections on the WRMSL in the sequence they are recovered to provide a consistent final MS data set. The STMSL is only and specifically used to measure selected sections from a new hole as needed, thus out of sequence, to provide rapid assessments and instructions to the driller.

Here comes the trick: In order to be able to correlate data from the STMSL with data from the WRMS in Correlator, the two data sets must be imported as two different data types, MSTL and STMSL, even though they are in fact both MS.
The Correlator Downloader application assists you with that: make sure you check the Split by instrument check box when downloading MS data, which results in two separate data files. When importing to Correlator, you assign the explicit custom names WRMSL STMSL for the two data files and you'll be able to correlate them.

3.2. Summary of Data Manager functions

When launching Correlator, you land on the Data Manager page. If you arrive there for the first time, you have only one function available:

• Root: right-click to add data

As soon as the first data file is imported, Correlator creates a data directory for the site, using the site name from the data file, and the following data group folders:

• One folder for each imported data type. Within each data type, the imported data are represented by a line item for each hole.
• The Section Summaries folder described above, which needs one file and list item per hole.
• The Saved Tables folder where Correlator saves affine and splice tables for the site.

Here, the functions available at each directory level are summarized for general reference (Table 3.2-1).

Table 3.2-1. Summary of data management functions.

3.3. Import primary correlation data

(Update)

(Correlator will not allow you to import two files for the same data type). The new data type labels will stick in the data type list for future use in your session.
Now you can correlate the STMSL data rapidly acquired on critical sections from a new hole to the WRMSL data from previous holes using the TIE and SET methods.
 However, because Correlator was not designed to correlate different data types (usually makes no sense) the affine table currently only keeps track of the SHIFT core, with the assumption that the REF core tie point is from the same data type as the SHIFT tie point. An arrow can therefore not be drawn.

3.2. Import core section summary files

(Update)

3.3. Import core section images

Important:

• You must first import some primary correlation data (and the associated section summary data) before you can import images.
• The images must be JPEG format and in the order of ~200 kb each, which is sufficient for most correlation purposes. Try higher resolution at your own risk.

To import images:

• Right-click on Images in the Data Manager window.
• Navigate to your local data directory
• Select the folder containing the JPEG images you want to import and click import.

3.4. Load data for correlation

(Update)

3.5. Update Correlator data

(Update)

3.6. Export data

Export affine and splice tables

(Update)

Export correlation data

(Update)

Export spliced data and images

(New)

4. Depth shift cores

4.1. Depth shift concepts

(Update)

4.2. Shift cores with the TIE method

(Update)

4.3. Shift cores with the SET method

(Update)

4.4. Undo shifts

(Update)

4.5. Manage affine tables

(Update)

5. Construct the splice

5.1. Splice concepts

(update)

5.2. Create a basic splice

(update)

5.3. Insert a core into an existing splice

(update)

5.4. Shifting cores that are already part of a splice

Concept and rules

Ideally, depth shifting should be concluded before a splice is assembled. However, life is not ideal and you may want to shift a core after you have constructed a splice. Because splice intervals are defined by their CCSF depths, shifting their cores invalidates the splice interval boundaries. A core shift results in a gap, an overlap, or both in the splice, depending whether a chain of tied cores or a single core are shifted, and whether the core(s) are shifted up or down. Correlator will deal with each situation in a clearly defined manner and provide you with relevant information.

Splice intervals boundaries are specified relative to the CCSF depth scale defined by the core sifts. The reason we select a certain core interval as being a splice interval is based on our interpretation of core quality based on the proxy data used to correlate the cores. The interval is therefore intrinsically defined by the the core-section-offset-in-section identity. When we shift cores we want to preserve that intrinsic splice interval. Therefore:

• Rule 1: Splice intervals are shifted with the cores they are associated with.
  • This inevitably creates at least one gap or one overlap in splice intervals.
• Rule 2: Overlaps in splice intervals are automatically resolved by 'clipping' the redundant part of the interval that is shifting.
  • We can do that because both overlapping intervals were approved as suitable for the splice and which one to use is typically a toss-up. You can always move the spice interval boundary very easily if that is not your desired solution.
  • The core-section-offset identity of the clipped splice interval boundary is reverse-computed from the CCSF depth to the CSF-A depth, from where the offset (cm) from top of section is obtained.
• Rule 3: Gaps created by the shifting of splice intervals are left open and you need to go and close them.
  • We prefer not to do that automatically because the interval needed to close the gap has not explicitly been assigned to the splice.

Example

Here is an example of your options and the program's responses when you shift cores that are part of a splice.

Case 1: Shift chain down

'This core and all related cores below' down

• You define a new tie from REF core A3 to SHIFT core B3 where the tie point in SHIFT core B3 is dz m shallower than the REF tie point in A3. This means the SHIFT core will shift down (Fig. X).
• You select “This core and all related cores below”
• The program asks for confirmation of the action (Fig. X), and assuming you OK it:
  • Replaces the previous tie with the new tie and shifts core B3 with all related cores from all holes that are deeper than REF core A3 downwards by dz m, maintaining all ties below the new one.
  • Shifts all splice intervals from from the shifting cores downwards by dz.
  • Creates a gap of length dz between splice intervals A3 and B3 and extends the CCSF scale by dz m.
• You cover the splice gap with a segment from either core A3 or B3, or partially with segments from both cores - it is up to you to decide.

Fig. X. Dialog for repairing a splice gap 

Figure X. Dialog for non-existent core interval. 

• • This solution is not valid, interval boundary falls outside the core.
  • Click the OK button and you return to the four options.

• The third option on the menu allows users to cover the gap with a combination of extensions from both splice intervals using the normal interactive splicing interface.
• The final option cancels the shift and nothing happens to cores or splice intervals.

Case 2: Shift chain up

'This core and all related cores below' up

• Define a new tie from REF core A3 to SHIFT core B3 where the tie point in SHIFT core B3 is dz m deeper than the REF tie point in A3. This means the SHIFT core will shift up (Fig. X).
• Select “This core and all related cores below” and click OK.
  • Results are analogous to those described in previous case.
• Because this action has shortened the CCSF scale, an overlap is created at the top of the splice interval representing the shifted cores. The overlap can be removed completely with a segment from either core A3 or B3, or partially with segments from both cores - it is up to you to decide. Correlator offers you the options in a pop-up window (Fig. X).

Figure X. Dialog for repairing a splice overlap. 

• • "This shift creates an overlap in the splice. How do you want to proceed?’
    • Clip the splice interval from core A3
    • Clip the splice interval from core B3
    • Leave the overlap and let me fix the splice manually
    • Cancel shift

• The first two options apply an “auto-fix”.  The calculated sample identity is validated and if the section-offset doesn’t actually exist in the same core, an error dialog is presented (Fig. X).

Figure X. Dialog for non-existent core interval.

• • This solution is not valid, interval boundary falls outside the core.
  • Click the OK button and you return to the four options.

• The third option on the menu allows users to remove the overlap with a combination of clipping parts from both splice intervals using the normal interactive splicing interface.
• The final option cancels the shift and nothing happens to cores or splice intervals.

Case 3: Shift single core down

'This core only' up

• Define a new tie from REF core A3 to SHIFT core B3 where the tie point in SHIFT core B3 is dz m shallower than the REF tie point in A3. This means the SHIFT core will shift down (Fig. X).
• Select “This core only” and click OK:
  • Replaces the previous tie with the new tie and shifts core B3 downwards by dz m.
• Because this action has not changed the total length of the CCSF scale, a gap is created at the top of the core and an overlap is created at the bottom of the core. The gap and overlap can each be repaired in two ways (see the first two cases) and Correlator therefore simply reminds you to do so yourself using the normal splice interface, or cancel the shift (Fig. X):

Fig. X. Dialog for repairing a single core splice gap and overlap.

• • "This shift creates a gap and an overlap in the splice. The splice interval associated with this core is therefore deleted and you need to splice it in again".
  • Cancel shift

Case 4: Shift single core up

'This core only' up

• Define a new tie from REF core A3 to SHIFT core B3 where the tie point in SHIFT core B3 is dz m deeper than the REF tie point in A3. This means the SHIFT core will shift up (Fig. X).
• Select “This core only” and click OK:
  • Replaces the previous tie with the new tie and shifts core B3 upwards by dz m.
• Because this action has not changed the total length of the CCSF scale, an overlap is created at the top of the core and a gap is created at the bottom of the core. The gap and overlap can each be repaired in two ways (see the first two cases) and Correlator therefore simply reminds you to do so yourself using the normal splice interface, or cancel the shift (Fig. X):

Fig. X. Dialog for repairing a single core splice gap and overlap.

• • "This shift creates a gap and an overlap in the splice. The splice interval associated with this core is therefore deleted and you need to splice it in again".
  • Cancel shift

5.5. Manage splice tables