Available translations

Data Mining - Datasets Type Conversions

foundations.png
Summary: Processes fetching data from the Internet are optimized for accessing a wide range of APIs available out there. The Dataset Type they use to store raw data is usually not the best one for data consumption within the system. For that reason Dataset Type conversions need to be done.
Types of Dataset Conversions
The type of conversions currently implemented within the system are:
How to Convert from One Dataset Type to Another
There are two ways to convert from one Dataset type to another.
  • Custom Code: Users can write code to handle the conversion in cases where the standard converter is not available for the type of conversion needed.
Custom Code Example
The Candles-Volumes indicator at the Master Data Mine converts One-Min candles and volumes into candles and volumes of 2 min, 3 min ecc.. up to 24hs. To do this, it runs a custom made algorithm that aggregates that information provided by the Exchange Raw Data Sensor Bot.
Standard Converter Example
The Asset Metrics Indicator Bot, defined at the Messari Data Mine, uses the Standard Converter from One-Min-Daily Dataset type to Multi-Time-Frame-Daily and Multi-Time-Frame-Market Dataset types. In order to use the converter, the bot's Process Definition nodes needs to have a framework of the converter defined within its config.
To convert to Multi-Time-Frame-Daily, set the framework name as follows: 
 {
    "codeName": "From-One-Min-To-Multi-Time-Frame-Daily",
    "normalWaitTime": 0,
    "retryWaitTime": 10000,
    "framework": {
        "name": "From-One-Min-To-Multi-Time-Frame-Daily"
    }
}
To convert to Multi-Time-Frame-Market, set the framework name like this: 
  {
    "codeName": "From-One-Min-To-Multi-Time-Frame-Market",
    "normalWaitTime": 0,
    "retryWaitTime": 10000,
    "framework": {
        "name": "From-One-Min-To-Multi-Time-Frame-Market"
    }
}
Previous
Data Mining - Dataset Types
Next
Data Mining - Indicator Frameworks

Data Mining — TOC

You just read page 4 in the topic.

1. Data Mining - Introduction

2. Data Mining - Workflows

3. Data Mining - Dataset Types

4. Data Mining - Datasets Type Conversions

5. Data Mining - Indicator Frameworks

6. Data Mining - Data Mine Hierarchy

7. Data Mining - Managing Execution

8. Data Mining - Data Dependencies

9. Data Mining - Status Reports

10. Data Mining - Status Dependencies

11. Data Mining - Deployments

12. Data Mining - Products

13. Data Mining - Plotting

14. Data Mining - Fetching Data From APIs

15. Data Mining - Consuming Data From Strategies

16. Data Mining - Indicator Bots

17. Data Mining - Indicator Bots - Process Definition

18. Data Mining - Indicator Bots - Product Definition

19. Data Mining - Indicator Bots - Data Mining Workflow

20. Data Mining - Indicator Bots - Coding the Indicator's Logic

21. Data Mining - Plotters

22. Data Mining - Plotters - Shapes

23. Data Mining - How to Build an Indicator - Video Series