...
https://github.com/corewall/correlator/releases/tag/4.0_r1
Figure 1-1. Correlator 4 splash screen.
Miscellaneous notes:
...
Figure 1-2. Correlator File menu.
2. Correlator basic functions overview
...
Correlator operates within a single window that can be expanded as much as available monitor space permits. When launching Correlator, you land on the Data Manager view (Fig. 2.1-1). You can toggle to the Display view by using the Go to Display button in the lower left corner of the window.
Figure 2.1-1. Data Manager view. Note the button in the lower left corner used to switch to the Display view.
The Data Manager view has two functional tabs across the top:
...
The function in the Data Manager view are described in the Manage data in Correlator section.
2.2. Display view
Display functional areas
The second and main view in Correlator is the Display view (Fig 2.-2-1). You can toggle back to the Data Manager view by using the Go to Data Manager button in the lower left corner of the window.
Figure 2.-2-1. Display view, with the Display Preferences tab open on the right. Note the button in the lower left corner used to switch to the Data Manager view.
The Display view has three functional areas:
- Left plot area for depth shifting
- Right plot area for splicing (can be toggled off in the Display Preferences tab if more display space is preferred for depth shifting, see below)
- Control function area panel on the far right.
The control function area has four tabs:
- Close: This tab acts like a button and closes the control panel. The control panel can be opened again by clicking any of the other four tabs.
- Shift Cores: This tab has controls for depth shifting, which are described in the Depth shift cores section.
- Splice Cores: This tab has controls for splicing, which are described in the Construct the splice section.
- Display Preferences: This tab has general data display controls (Fig. 2.2-2)and is described below.
- Data Filters: This tab offers data filtering options, including decimate, smooth and cull (Fig. 2.2-3), and is described below.
Data Filters tab
Correlator offers three fundamental options to filter the data for the purpose of displaying them (Fig. 2. 2-2a3a). The original data are still kept in memory and can be displayed, either in addition to the filtered data, or again by deleting the filtersfilter.
To filter the data:
- Select the data type you want to filter (Fig. 2.2-2b3b).
- To decimate the selected data type, click the Edit button in the Decimate area (Fig. 2.2-2c3c).
- Enter a number in the Show every <N> points to limit the display to every Nth data point.
- Click the Apply button.
- To smooth the selected data type, click the Edit button in the Gaussian Smoothing area (Fig. 2. 2-2d3d).
- Select the type of rolling window: Points or Depth (cm).
- Enter the Width in points or cm.
- Select the Display option: Smoothed only or Original & Smoothed
- Note: you can change the color of the smoothed trace in the Display Preferences > Set Colors
- Click the Apply button
- Select the type of rolling window: Points or Depth (cm).
- To cull the selected data type, click the Edit button in the Cull area (Fig. 2. 2-2e3e). You have to cull type options:
- The first type of cull is to Cull data from sample edges.
- Enter the interval in cm in Cull <x> cm from core tops.
- The second type is to Cull outliers.
- Enter a value for Cull data values > x
- Enter a value for Cull data values < x
- Click the Apply button.
- The first type of cull is to Cull data from sample edges.
Figure 2. 2-23. Three data filtering options on the Data Filter tab: Decimate, Gaussian Smoothing, and Cull. Shown is a 9-point Gaussian filter applied to the magnetic susceptibility data whereby both original and smoothed (white trace overlay) are plotted. The filters can be edited or deleted.
a
b
c
d
e
Note:
- Data filters are applied to the data from all cores for a specified data type. For a more specific filtering of known intervals with severe core disturbance, etc., you have the option to apply a file with those intervals specified in the Correlator Downloader application, at the time of data download from the LIMS database.
- At this time, data culling can only be specified for top of cores (to remove data from “exotic” material washed down from higher up in the hole). A future version may include culling from core bottoms (“exotic” material “sucked in” when the piston core was removed) and culling from section ends (“edge effect”).
- Some odd behavior has been observed when deleting a cull filter under certain circumstances, and this has not yet been repaired. An easy workaround is to re-load the data.
...
This tab offers numerous display options that apply to core shifting and splicing functions (Fig. 2(Fig. 2-3a4a). Two new features include the ability to change the display order of the data types, including not showing them (Fig. 2.2-3b) and the ability not to display the data for selected holes (Fig. 2.2-3c). Data ranges can be set numerically for each data type (Fig. 2.2-3d) and visually be adjusting track widths. The color panel is updated so you can adjust the scheme to your liking. Additional check boxes were added to toggle on or off the display of lines, arrows and labels. These options should all be self-explanatory. Load some data, shift some cores, make some splices, and then simply explore the options be clicking around - your selections will not affect the data.
Figure 2.2-3. Display preferences.
a b
c
You can:
- Show or hide the splice partition with the check box at the top (Fig. 2-4)
- Show or hide the loaded data types and change their display order (Fig. 2-4b)
- Show or hide the loaded data by hole (Fig. 2-4c)
- Overlap data traces for all holes by data type
- Set a depth range to be displayed in the window and zoom in and out
- Set the data value ranges by data type (Fig. 2-4d)
- Choose whether to show data extending beyond the track width
- Set the track width (separately for data traces and images)
- Toggle on and off several line and arrow options
- Change the line colors
Figure 2-4. Display preferences.
a 
b 
c 
d 
e 
d e
2.3. Application menus
The application menus are typically not needed. Here we present only Only the View menu as is shown here because it has a few low-level display options not available in the Display Preferences tab of the Display view (Fig. 2.3-15).)
Figure 2.3-15. View menu.
3. Manage data in Correlator
3.1. Prepare correlation data
Data directories
How you manage your data directory that Correlator is relatively flexible in terms of data file content and formats. Its import functions allow you to specify the only columns it really requires: depth and data. However, having the data files in a consistent format makes the process simpler and reliable. The following are the recommendations should be considered to be best practices on the JR.
...
- Batch-download the cropped section half images for a hole into a separate folder.
- Use the R script provided to crop and reduce the images, then move the reduced images to the images folder in the correlation data site folder.
- A separate user guide describes how to use the R script.
uses to import and update data is partly a function of your overall workflow and partly personal preference. The recommendation provided here is to keep site folders near the top of your directory, with each site folders having a data file for each data type and each hole, except for images, where the section half images are collected in hole folders (see example in Fig. 3-1). If working with JR data stored in the LIMS database:
- Download the non-image data using the Correlation Downloader app and place them directly in the site folder.
- The Correlation Downloader app has its own user guide, but one important feature is highlighted here: when you are correlation 'in real time', you will add data iteratively to the hole file for each core that is recovered and curated. Make sure you check the Append core check box.
- Download images using the LORE app and follow the current 'workaround' procedure:
- Batch-process the high-resolution, cropped JPG images with an R script (a separate user guide describes how to use the R script)
- Place the folder with the reduced images in the site folder.
If you have existing affine and splice tables for the site, place them in the site folder as well.
If working with other data, all you you have to make is that they have a depth column and a value column, and place the in the site folder.
Figure 3.1-1. Example of a site folder with correlation data, hole files for non-image data and hole folders for images.
Special case: multiple files for the same data type
One of the primary objectives of stratigraphic correlation is to correlate section data from a new hole rapidly to the data from one or more previous holes to check if coring gaps in previous holes are recovered in the current hole. If coring gaps are not covered adequately due to the change in water depth (mostly due to tides, but also other environmental and operations factors), you want to provide relative coring offset (drill down interval) instructions to the drillers in near-real time.
In order to achieve this objective, we implemented a special task multisensor logger (STMSL), also referred to as ‘fast track’, with the same type of magnetic susceptibility (MS) instrument as the one mounted on the primary whole-round multisensor logger (WRMSL). The two instruments are the same for all practical purposes, however, users have found that slight differences in the data may exist (normally negligible compared to the overall analytical error). In any case, the ultimate goal is to measure all sections on the WRMSL in the sequence they are recovered to provide a consistent final MS data set. The STMSL is only and specifically used to measure selected sections from a new hole as needed, thus out of sequence, to provide rapid assessments and instructions to the driller.
Here comes the trick: In order to be able to correlate data from the STMSL with data from the WRMS in Correlator, the two data sets must be imported as two different data types, MSTL and STMSL, even though they are in fact both MS.
The Correlator Downloader application assists you with that: make sure you check the Split by instrument check box when downloading MS data, which results in two separate data files. When importing to Correlator, you assign the explicit custom names WRMSL STMSL for the two data files and you'll be able to correlate them.
3.2. Summary of Data Manager functions
When launching Correlator, you land on the Data Manager page. If you arrive there for the first time, you have only one function available:
- Root: right-click to add data
As soon as the first data file is imported, Correlator creates a data directory for the site, using the site name from the data file, and the following data group folders:
- One folder for each imported data type. Within each data type, the imported data are represented by a line item for each hole.
- The Section Summaries folder described above, which needs one file and list item per hole.
- The Saved Tables folder where Correlator saves affine and splice tables for the site.
Here, the functions available at each directory level are summarized for general reference (Table 3.2-1).
Table 3.2-1. Summary of data management functions.
...
Function (Menu item)
...
Function description
...
Root level
...
Site level
...
Data group items
...
Data items
When launching Correlator, you land on the Data Manager page. If you arrive there for the first time, you have only one function available:
- Root: right-click to add data
As soon as the first data file is imported, Correlator creates a data directory for the site, using the site name from the data file, and a number of data type folders that are explained in the Add new data section. An example corresponding the user directory example is shown in Fig. 3-2. The Correlator directory displays in the hierarchical order Site/Data type/Hole, sorted alphabetically at each level.
Figure 3.1-2. Data Manager view of the imported data.
About data types
Note that all data types that can be imported into Correlator are not managed equally:
- Regular data types in this example:
- MS STMSL
- MS WRMSL
- NaturalGamma
- Susceptibility
- Data types handled in special ways (see next section):
- Images
- Saved Tables
- Section Summaries
This is explained further in subsequent sections.
3.2. Summary of Data Manager functions
You can access numerous data management functions from context menus (right-click) at each directory level. Table 3-1 lists all the functions and tabulates the directory levels and data types where they are accessible. Figure 3-3 shows the corresponding menu windows.
Table 3.2-1. Summary of data management functions.
Function (menu item) | Function description | Root level - no data yet (Fig. 3.2-1a) | Site level - most used (Fig. 3.2-1b) | Non-image data type level (Fig. 3.2-1c) | Image data type level (Fig. 3.2-1d) | Saved Tables (Fig. 3.2-1e) | Site Summaries (Fig. 3.2-1f) |
Add new data | Import data from directory | YES | YES | YES | |||
Load | Load data for plotting in Display view. | YES | YES | YES | |||
Update | The specified file is re-imported and the information in Correlator is updated. | YES | YES | YES | |||
Delete | Remove from the Correlator database. | YES | YES | YES | |||
Disable/Enable | Disable or Enable (should read “Enabled” and “Disabled”) status determines whether the data are loaded and can be plotted in the Display view. | YES | YES | ||||
Export | Mainly used to generate affine and splice tables as CSV files that can be uploaded to LIMS. | YES | YES | ||||
View | Brings up a modal window with the data in a grid. | YES | |||||
Import affine table | Import a previously saved affine table so it could be re-applied to the data. | YES | |||||
Import splice table | Import a previously saved splice table so it could be re-applied to the data. | YES | |||||
Import legacy affine table | Only used if you want to apply an old affine table to the currently imported data. (You would need to know the difference...) | YES | |||||
Import Section Summary Files(s) | Only used if section summaries are not automatically loaded from Correlation Downloader target directory, which is the standard procedure now! | YES |
Figure 3.2-1: Context menus at different directory levels (see Table 3-1).
a 
b 
c 
d 
e 
f 
3.3. Add new data (non-image data types)
General case
To import new data files:
- If no data exist yet in the Correlator database, the Data Manager window is empty except for the Root item.
- Right-click on Root to get the single item on the menu, Add new data (Fig. 3.2-1a).
If non-image data exist in the directory, Correlator has created a directory with at least one Site folder.
Right-click on the top item on the menu, Add new data (Fig. 3.2-1b)
In either case, selecting Add new data opens a browser window (Fig. 3.3-1).
Browse to the data files in your directory - the path will stick for future imports and updates.
Select a single file, or multiple files of the same data type (and the same format).
In the example in Fig. 3.3-1, the natural gamma ray data files for Holes 361-U1476A, B and D are selected.
The first 30 rows of data for each hole are displayed in the Generic Data tab (Fig. 3.3-2).
For commonly used data types, all column headers and column data should be automatically and correctly populated.
If that is not the case, you get a warning and must specify at least the Depth and the Data columns using the correct selection from the column header context menu.
Click the Import button at the bottom right and the focus returns to the Data Manager tab where a data summary line is added).
Figure 3.3-1. The Import button opens in Generic Data window, where user may have to fill in the Data Type column by clicking on the header “Data Type” and selecting the appropriate item (in this case, Natural Gamma).
Figure 3.3-2. Upon importing data, line items for each data type and each hole are added in the Data Manager tab . Note the Saved Tables folder created for affine and splice tables later created. Also note the Section Summaries folder automatically created and populated with the section summaries from the LIMS registry (more on that below).
Special case: multiple files for the same data type
Correlator is trying to help you with file naming upon import. However, sometimes you may need to help Correlator learn the import data file, such as when you use correlation data from sources other than the JRSO Downloader app.
One of the primary objectives of stratigraphic correlation is to correlate section data from a new hole rapidly to the data from one or more previous holes to check if coring gaps in previous holes are recovered in the current hole. If coring gaps are not covered adequately due to the change in water depth (mostly due to tides, but also other environmental and operations factors), you want to provide relative coring offset (drill down interval) instructions to the drillers in near-real time.
In order to achieve this objective, we implemented a special task multisensor logger (STMSL), also referred to as ‘fast track’, with the same type of magnetic susceptibility (MS) instrument as the one mounted on the primary whole-round multisensor logger (WRMSL). The two instruments are the same for all practical purposes, however, users have found that slight differences in the data may exist (normally negligible compared to the overall analytical error). In any case, the ultimate goal is to measure all sections on the WRMSL in the sequence they are recovered to provide a consistent final MS data set. The STMSL is only and specifically used to measure selected sections from a new hole as needed, thus out of sequence, to provide rapid assessments and instructions to the driller.
Here comes the trick: In order to be able to correlate data from the STMSL with data from the WRMS in Correlator, the two data sets must be imported as two different data types, MSTL and STMSL, even though they are in fact both MS.
The Correlator Downloader application assists you with that: make sure you check the Split by instrument check box when downloading MS data, which results in two separate data files. When importing to Correlator, you assign the explicit custom names WRMSL STMSL for the two data files and you'll be able to correlate them
...
Add new data
...
Import data from directory
...
YES
...
YES
...
YES
...
Load
...
Load data for plotting in Display view.
...
YES
...
YES
...
YES
...
Update
...
The specified file is re-imported and the information in Correlator is updated.
...
YES
...
YES
...
YES
...
Delete
...
Remove from the Correlator database.
...
YES
...
YES
...
YES
...
Disable/Enable
...
Disable or Enable (should read “Enabled” and “Disabled”) status determines whether the data are loaded and can be plotted in the Display view.
...
YES
...
YES
...
Export
...
Mainly used to generate affine and splice tables as CSV files that can be uploaded to LIMS.
...
YES
...
YES
...
View
...
Brings up a modal window with the data in a grid.
...
YES
3.3. Add new data
Basic correlation data
To import data files:
- Right-click on the top item in the Data Manager window and select Add new data.
- A browser window opens that allows you to browse to the data files in your directory of choice (the path will stick for future imports and updates).
- Select a single file, or multiple files of the same type (and format). In the example in Fig. 3.2-1, magnetic susceptibility was selected for holes 361-U1476A and 361-U1476B.
- The first 30 rows of data for each hole are displayed in the Generic Data tab (Fig. 3.2-1).
- For commonly used data types, all column headers and column data should be automatically and correctly populated.
- If that is not the case, you get a warning and must specify at least the Depth and the Data columns.
- Click the Import button at the bottom right and the focus returns to the Data Manager tab where a data summary line is added (Fig. 3.2-2).
Figure 3.2-1. The Import button opens in Generic Data window, where user may have to fill in the Data Type column by clicking on the header “Data Type” and selecting the appropriate item (in this case, Natural Gamma).
Figure 3.2-2. Upon importing data, line items for each data type and each hole are added in the Data Manager tab . Note the Saved Tables folder created for affine and splice tables later created. Also note the Section Summaries folder automatically created and populated with the section summaries from the LIMS registry (more on that below).
Correlator is trying to help you with file naming upon import. However, sometimes you may need to help Correlator learn the import data file, such as when you use correlation data from sources other than the JRSO Downloader app.
One JR workflow, however, requires you to help a little even when using the Correlation Downloader. As described in section 3.1, you may have split the MS data into two files based on which core logger they were obtained from. What happens upon import depends to some degree on the file naming:
...