Available translations

Data Mining - Datasets Type Conversions

foundations.png
Summary: Processes fetching data from the Internet are optimized for accessing a wide range of APIs available out there. The Dataset Type they use to store raw data is usually not the best one for data consumption within the system. For that reason Dataset Type conversions need to be done.
Types of Dataset Conversions
The type of conversions currently implemented within the system are:
How to Convert from One Dataset Type to Another
There are two ways to convert from one Dataset type to another.
  • Custom Code: Users can write code to handle the conversion in cases where the standard converter is not available for the type of conversion needed.
Custom Code Example
The Candles-Volumes indicator at the Master Data Mine converts One-Min candles and volumes into candles and volumes of 2 min, 3 min ecc.. up to 24hs. To do this, it runs a custom made algorithm that aggregates that information provided by the Exchange Raw Data Sensor Bot.
Standard Converter Example
The Asset Metrics Indicator Bot, defined at the Messari Data Mine, uses the Standard Converter from One-Min-Daily Dataset type to Multi-Time-Frame-Daily and Multi-Time-Frame-Market Dataset types. In order to use the converter, the bot's Process Definition nodes needs to have a framework of the converter defined within its config.
To convert to Multi-Time-Frame-Daily, set the framework name as follows:
 {
    "codeName": "From-One-Min-To-Multi-Time-Frame-Daily",
    "normalWaitTime": 0,
    "retryWaitTime": 10000,
    "framework": {
        "name": "From-One-Min-To-Multi-Time-Frame-Daily"
    }
}
To convert to Multi-Time-Frame-Market, set the framework name like this:
  {
    "codeName": "From-One-Min-To-Multi-Time-Frame-Market",
    "normalWaitTime": 0,
    "retryWaitTime": 10000,
    "framework": {
        "name": "From-One-Min-To-Multi-Time-Frame-Market"
    }
}
Previous
Data Mining - Dataset Types
Next
Data Mining - Indicator Frameworks