FluxArchitectures has finally been published as a Julia package! Installation is now as easy as typing ] to activate the package manager, and then

add FluxArchitectures

at the REPL prompt.

The next model in the FluxArchitectures repository is the Temporal Pattern Attention LSTM network based on the paper “Temporal Pattern Attention for Multivariate Time Series Forecasting” by Shih et. al.. It claims to have a better performance than the previously implemented LSTNet, with the additional advantage that an attention mechanism automatically tries to determine important parts of the time series, instead of introducing parameters that need to be optimized by the user.

The next model in the FluxArchitectures repository is the “Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction”, based on the paper by Qin et. al., 2017. It claims to have a better performance than the previously implemented LSTNet, with the additional advantage that an attention mechanism automatically tries to determine important parts of the time series, instead of introducing parameters that need to be optimized by the user.

The first model in the FluxArchitectures repository is the “Long- and Short-term Time-series network” described by Lai et al., 2017.

It’s been a while since anything happened on this blog. So what happened in the meantime? Well, for the first I abandoned Tensorflow.jl in favour of Flux.jl. It seems to be the package that most people are using these days. It has a nice way of setting up models, and is nicely integrated into other parts of the Julia ecosystem as well (say, for example by combining it with differential equations to give scientific machine learning).

Finally, all files in the GitHub repository have been updated to be able to run on Julia 1.1. In order to be able to run them (at the time of writing), the developmental versions of the Tensorflow.jl and PyCall.jl packages need to be installed. Some notable changes are listed below:

This is the final exercise of Google’s Machine Learning Crash Course. We use the ACL 2011 IMDB dataset to train a Neural Network in predicting wether a movie review is favourable or not, based on the words used in the review text.

In the last post, we obtained the files test_data.h5 and train_data.h5, containing text data from movie reviews (from the ACL 2011 IMDB dataset). In the next exercise, we need to access a one-hot encoded version of these files, based on a large vocabulary. The following code converts the data and stores it on disk for later use. It takes about two hours to run on my laptop and uses 13GB of storage for the converted file.

The last exercise of the Machine Learning Crash Course uses text data from movie reviews (from the ACL 2011 IMDB dataset). The data has been processed as a tf.Example-format and can be downloaded as a .tfrecord-file from Google’s servers.

In this exercise, we look at the famous MNIST handwritten digit classification problem. Using the MNIST.jl package makes it easy to access the image samples from Julia. Similar to the logistic regression exercise, we use PyCall and scikit-learn’s metrics for easy calculation of neural network accuracy and confusion matrices.

This is the last exercise that uses the California housing dataset. We investigate several possibilities of optimizing neural nets:

In this exercise, we construct a neural network with a given structure of the hidden layer.

The next programming exercise in the machine learning crash course is about L1-regularization and sparsity.

The next part of the Machine Learning Crash Course deals with Logistic Regression.

The next part of the Machine Learning Crash Course deals with constructing bucketized features and feature crosses. The Jupyter notebook can be downloaded here.

The fourth part of the Machine Learning Crash Course deals with finding a minimal set of features that still gives a reasonable model.

The third part of the Machine Learning Crash Course deals with validation of the model.

In this second part, we create a synthetic feature and remove some outliers from the data set.

The first exercise is about getting a general idea of TensorFlow. We run through the following number of steps:

With this blog, we are going to explore the Tensorflow.jl-Package for the Julia Programming Language.