Data Preparation #2

Edit: I just fixed the .java file by adding bw.flush() and bw.close() to the code. This fixed the remaining rows problem and now any data is almost ready to be used! Next: creating an M-script to prepare a ready matrix straight out of a .dat or .csv file!


I’ve managed to prepare an almost-ready matrix sample that can be tested in a neural network. It contained 4 columns for OHCL prices, one column for RSI, one column for MACD, two columns for SMA(10) and SMA(20), and finally the last column contains either 0 or 1, with 1 indicating that a moving average crossing happened in this bar.

There was 213 crossings in 3414 bars of hourly USDCAD (CAD = Canadian Dollars) prices, which originated from around 5000 unclean bars. That means that I will roughly need to use 20,000 bars to obtain a sample of around 1000 crossings and targets. 20,000 unclean hourly bars represent a little more than 2 years and three months. This is highly available.

What’s left to do is to add one more column where by next to every “1” in the latest column is the value of either the higest high (if this was an upcrossing) or lowest low (if this was a downcrossing).

The java application that cleans the data up and prepares it to be loaded into a fints is ready with only a minor problem, it does not load the last few rows correctly. I will look into it again tomorrow and try to get the matrix ready to be loaded.


class prep {

public static void main (String args[]){

String currentline;
int k=0;
int g=0;

try {

BufferedReader br = new BufferedReader(new FileReader(args[0]));
BufferedWriter bw = new BufferedWriter(new FileWriter(args[1]));


while ((currentline = br.readLine())!= null) {

if(currentline.contains("\t0\t") || currentline.contains("DATE"))


else {

catch (IOException e) {
System.err.println("Error: " + e);


Below is a screenshot of the fints data window with the USDCAD prices matrix loaded.

Screenshot-Financial Time Series Tool

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s