Superalgos Data Mining is the industrial-scale research and numbers-crunching product of the Superalgos Platform. On the surface, the data processing features are similar to those in Superalgos Markets Research. However, the product's vocation diverges significantly.
While Superalgos Markets Research is a robust tool to produce all sorts of data processing that may result in customized studies and various forms of reports, Superalgos Data Mining has a focus on an industrial-scale deployment for the data mining operations of the future.
Superalgos Data Mining helps running such a business, coordinating the data processing operation on scalable data-mining farms, and—most importantly—administering the business, managing subscription services exposed over a decentralized intelligence network.
In the near future, trading intelligence will be distributed across the Superalgos Network, a decentralized peer-to-peer network of nodes running as different products in the Superalgos Platform. Part of this collective intelligence will be open source, originating in multiple spontaneous collaborations of people and machines around the globe. Another part of this intelligence will originate in private companies specializing in producing markets analysis and trading intelligence for profit.
Superalgos Data Mining is one of the products in the Superalgos Platform targeting such for-profit operations. Intelligence across the Superalgos Network will be consumed by nodes in the network via subscription services. While the Superalgos Network hasn't been deployed yet, the roadmap of Superalgos Data Mining includes the support for managing the whole operation, including the subscription services through which companies will expose their intelligence to the network.
The architecture of the infrastructure behind Superalgos Data Mining, storing highly fragmented, standardized flat files for each time frame, product/study, market, data source/exchange, responds to the scalability demands of major data-mining operations. Processes' architecture is crucial too, as multiple processes may run in sync with each other. As such, datasets may be processed in grawing layers of nested calculations, producing ever more elaborate data crossings.