In this comprehensive course on algorithmic trading, you will learn about three cutting-edge trading strategies to enhance your financial toolkit. In the first module, you'll explore the Unsupervised Learning Trading Strategy, utilizing S&P 500 stocks data to master features, indicators, and portfolio optimization. 

Next, you'll leverage the power of social media with the Twitter Sentiment Investing Strategy, ranking NASDAQ stocks based on engagement and evaluating performance against the QQQ return. Lastly, the Intraday Strategy will introduce you to the GARCH model, combining it with technical indicators to capture both daily and intraday signals for potential lucrative positions.

✏️ Course developed by @lachone_ 

💻 Code and course resources: https://github.com/Luchkata/Algorithmic_Trading_Machine_Learning/tree/main

🔗 Learn more about Lachezar and Quantitative Trading with Python here: https://www.quantfactory.ai/p/become-a-quant-trader1

⭐️ Contents ⭐️
0:00:00 - Algorithmic Trading & Machine Learning Fundamentals
0:15:25 - Building An Unsupervised Learning Trading Strategy
2:05:08 - Building A Twitter Sentiment Investing Strategy
2:28:08 - Building An Intraday Strategy Using GARCH Model

🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 南宮千影
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan 
👾 Oscar Rahnama

--

Learn to code for free and get a developer job: https://www.freecodecamp.org

Read hundreds of articles on programming: https://freecodecamp.org/news

In this comprehensive course on algorithmic trading, you will learn about 3 cutting edge trading strategies to enhance your financial toolkit.

He is an experienced quantitative researcher and data scientist.

In the first module, you'll explore the unsupervised learning trading strategy, utilizing SP500 stocks data to master features, indicators, and portfolio optimization.

Next, you'll leverage the power of social media with the Twitter sentiment investing strategy, ranking Nasdaq stocks based on engagement and evaluating performance against the QQQ return.

Lastly, the intraday strategy will introduce you to the GARCH model, combining it with technical indicators to capture both daily and intraday signals for potential lucrative positions.

Hello and welcome to this free CodeCamp course on Algorithmic Trading, Machine Learning and Quant Strategies with Python.

My name is Vlad Cezar and I'll be your instructor throughout the course.

And in this course we are going to develop 3 big quantitative projects from start to end.

And the course overview will be the following.

First we are going to talk about algorithmic trading basics.

Then we're going to talk about machine learning in trading, some obstacles and challenges we may face while using machine learning in trading, then we are going to develop the first project which will be an unsupervised learning trading strategy using stocks from S&P 500.

The next project would be Twitter sentiment and it would be using data from Nasdaq 100 stocks and the third 1 would be focusing on 1 asset.

It would be an intraday strategy using a GART model to predict the volatility.

And after that we are gonna have a quick wrap-up and that will finish the tutorial.

But before we continue I would like to mention that this tutorial should not be construed as a financial advice.

It is for educational and entertainment purposes only.

We are going to develop some concepts and come up with strategies in the end.

But it's not a financial advice and you shouldn't make any decisions based on it.

As well for this course I would assume that you have at least some Python knowledge and understanding because down the road we'll deal with some complex problems and if you are new to Python you may get bogged down.

Nevertheless you learn some very interesting concepts So stay along and let's get into it.

It is trading on a predefined set of rules which are combined into a strategy or system.

It is developed by a programming language and it is run by the computer.

It can be used for both manual and automated trading.

By manual what I mean is that you may have a screener which comes up with a set of stocks you want to trade on a given day or you may have an algorithmic strategy which is developed into an alert system and whenever the conditions are triggered you get an alert but you execute it manually.

On the other hand, you may have a completely automated complex system which does a lot of calculation, comes up with positions and sizing, and then executes the trade automatically.

Okay, so what is the role of Python in algorithmic trading?

Python is the most popular language used in algorithmic trading, quantitative finance, and data science.

And this is mainly due to the vast amount of libraries that are developed in Python, as well the ease of use of Python.

It is mainly used for data pipelines, research, backtesting strategies, as well it can be used to automate strategies but Python is a slow language and it can be used to automate low complexity systems.

If you have a really high-end system which is really complicated and it needs to execute trades really quickly you would use Java or C++ for those strategies.

Algorithmic trading is a great career opportunity.

There are a lot of jobs with hedge funds, banks, prop shops and I've just checked the average yearly base salary for a quant researcher is around $173,000 and this is not including the yearly bonds.

It is a great career opportunity and if you're interested into it the main things you need to know are Python, you need to know how to backtest strategies, you need to know how to replicate papers, and you need to know machine learning in trading.

If you're interested into it, I'll definitely advise you to go for it.

Okay, so let's move on and talk a little bit about machine learning in trading and some use cases of machine learning.

When we talk about supervised learning we can use it for signal generation through prediction.

For example, we can come up with buy or sell signals on a given stock or a given asset based on predicting the return or the sign of the return of that asset.

We can as well use it in risk management.

For example, we may use a prediction to determine the position sizing or the weight of a given stock in our portfolio or to predict where exactly should our stop loss be.

And with unsupervised learning, we can use it to extract insights from the data.

For example, we can discover patterns, relationships, or structures within the data.

For example, clusters and use it in this way to help our decisions.

What are some of the challenges that we may face while trying to apply machine learning in trading?

And the first theoretical challenge we may face is the so-called reflexivity feedback loop.

And it is referring to the phenomenon that if we, for example, have a machine learning model that is predicting that a stock is going to go up each Friday and we can form a strategy around to profit from that phenomenon.

For example, we are going to buy each Thursday and then sell on Friday to capture that price increase move.

If we find this strategy throughout predictions and start trading it, with time other market participants as well are going to find this market phenomenon and start exploiting it as well, which would cause the price to start going up on Thursday because everybody is buying now on Thursday instead on Friday and then this strategy is going to be arbitrage away.

So it's this reflexivity feedback loop which is making predictions quite hard.

What is most hard while applying machine learning?

The most hard thing is to predict returns and predict prices.

The next quite hard thing to do is to predict return signs or the direction of a given asset.

The next thing is to predict an economic indicator.

For example, it is quite hard to predict non-farm payrolls or weekly jobless claims.

And a thing which is not that hard or quite straightforward is to predict the volatility of a given asset.

Furthermore, there are some technical challenges like overfitting a model or generalization.

Overfitting is that the model is learned the trained data too well and it fails on the test data and generalization is that the model is not performing the same as on the real data.

As well we may have non-stationarity in our training data and regime shifts which may ruin the performance of the model.

And the last thing is that if we have a really complicated model or a neural network it is like a black box and we are not able to interpret it correctly.

What is the usual workflow process in algorithmic trading and machine learning?

That would be to collect and prepare the data, then develop a hypothesis for a strategy, then you have to code the model and train the model, and then finally backtest the strategy.

Okay guys and some key takeaways from this course.

So you learn high-level concepts in quantitative finance as well practical machine learning in trading.

You will develop a project from idea to backtest final results.

However, we will not automate or execute any trades.

This course is all about developing a strategy from start to end so you see the workflow.

And this is just a purely research project for educational purposes.

I repeat it should not be considered as any financial advice whatsoever.

And with that we can move to the first project and let's get into it.

Okay so first project is about unsupervised machine learning trading strategy.

We are going to use data from S&P 500 stocks.

Let's talk a little bit about unsupervised learning in trading.

So it involves machine learning techniques to analyze financial data and discover patterns, relationships, and structures within this data without predefined labels or target variable.

Unlike supervised learning where the model is trained to make predictions, unsupervised learning focuses on extracting insights from the data.

Some use cases would be clustering, which we are going to use in the first project, dimensionality reduction, anomaly detection, market regime detection, and portfolio optimization.

In this project, what we are going to do is first we are going to download all the prices data for all S&P 500 stocks.

Then we are going to calculate different technical indicators and features for each stock.

Next, we are going to aggregate on a monthly level and filter only the top 150 most liquid stocks from the S&P 500 for each month.

Next, we are going to calculate monthly returns for different time horizons to add up to the features.

And the next step would be to download the Fama French factors and calculate rolling factor betas for each stock as well to add to the feature set.

And at this point we have enough features to fit a model and either make predictions or in our case we are going to fit the K-Means clustering algorithm, an unsupervised learning model and we will use it to group the stocks into similar assets into clusters and from those clusters then we will be able to for each month select stocks from a given cluster and we are going to analyze the clusters and select a particular cluster and then for Each month we are going to select those stocks within this cluster and form portfolios.

However, these portfolios will be optimized so the weights of the stocks within the portfolio we are going to find them by using the Efficient Frontier Max Sharpey Ratio portfolio weights.

And then we are going to form the portfolio, hold for 1 month and rebalance at the end of the month and form another Max Sharpe E-Ratio portfolio.

In the end, we'll have the strategy returns for each day and we will be able to compare our portfolio strategy returns to the S&P 500 returns themselves.

Actually, not small limitation is that we're going to use the most recent S&P 500 stocks list, which means that there may be a survivorship bias in this list.

Actually, In reality, you should always backtest strategies using survivorship-free bias data.

It is the condition when a stock which have actually went out of the S&P 500 because it was failing is currently not in the list, right?

So last year, for example, there was a stock which was failing and going down, down, down.

So at some point in December last year, they removed it from S&P 500 and they included a new stock.

So if we have made the optimization last November, we could end up with having this stock into our portfolio and actually it affecting our portfolio results.

But if we use the most recent S&P 500 symbols list, this stock would not be there.

And for the given project, we are not going to deal with this survivorship bias so the list we are going to work with has survivorship bias most probably so yeah that is a limitation you need to know.

In the second project we are going to develop a Twitter sentiment-based investing strategy.

We are going to use the Nasdaq 100 stocks and Twitter sentiment data.

This approach focuses on analyzing how people feel about certain stocks, industries or the overall market.

It assumes that the public sentiment can impact stock prices and for example if many people are positive about a particular company on Twitter it might indicate potential for that company stock to perform well.

What we are going to do first we're going to load the Nasdaq stocks Twitter sentiment data.

Then we're going to calculate a quantitative feature of the engagement ratio in Twitter for each stock.

After that we're going to rank all the stocks cross-sectionally for each month and create an equal weight portfolio.

In the end, we're going to compare the return of this portfolio to the Nasdaq itself.

The second strategy is much smaller than the first strategy.

There is no machine learning modeling in this strategy, but the idea here is to show you how alternative or different data, in this case sentiment data, can help us to create a quant feature and then create potential strategy out of it.

And the third project is about an intraday strategy using a GARC model.

In this 1 we are going to focus on a single asset and we'll be using simulated data.

Actually we'll have daily data and intraday five-minute data.

This approach involves buying and selling financial assets within the same trading day to profit from the short-term price movements.

Intraday traders usually use technical analysis, real-time data, and different risk management techniques to make decisions and profit from the strategies.

What exactly we are going to do in this project?

First, we are going to load the simulated daily data and the simulated five-minute data.

Then we are going to define a function which would fit in a rolling window, a GARC model to predict 1 day ahead volatility of the asset.

After we have that, we'll calculate prediction premium.

So we'll predict the volatility and we'll calculate the prediction premium and form a daily signal from it.

After we have calculated that, then we'll merge the daily data with the intraday data and calculate intraday technical indicators to form intraday signal.

So we will have a daily signal and then on top an intraday signal, so 2 signals.

And after we have that, we will generate position entry and hold until the end of the day.

In the end we are going to calculate the final strategy returns and this is it for the third project.

The idea is to show you how predicting volatility works in intraday strategies.

And yeah with that guys we are ready to jump into the first project so let's get into it.

Okay let's start with the first project the unsupervised learning trading strategy but before we continue with the coding guys the first step would be for you to pause the video and install all the needed packages for this project so I prepared a small list up here with all the packages we are going to need for the project So those are pandas, numpy, matplotlib, statsmodels, pandas data reader, datetime, yfinance, sklearn and piportfolio.

You can open an anaconda prompt like that, right?

And then just write pip install and the package name pip install pandas for example and do that for each package that's 1 way another way to do it is through the notebook itself you can type the following so pip install and then the package name so pip install pandas you do ctrl shift and run the cell and it will install the package.

Pause the video take your time install all the packages and let's continue.

Okay so the first step would be to download the S&P 500 constituents prices data But before that we'll have to import all the packages we'll use throughout this project.

Those are all the packages we've just installed and now we are importing into the Jupyter notebook.

The next step would be to download S&P 500 constituents data.

To do that we can go to this link on Wikipedia and up here you can see that they have a table containing S&P 500 component stocks like the symbol, the name of the security sector, industry, date added as well.

So we would like to load this table into our Jupyter notebook.

We'll just call a pandas read HTML function and let's see what this would return us.

Okay, so this is returning a list containing 2 elements.

So, actually we are interested in the first data frame.

We will assign that to an object called SMP500.

And the next step would be to grab this symbol column and extract all the symbols in a list.

But before that, I think we would have to make a little cleaning on some of the symbols because I know that 1 or 2 of the symbols contain a dot and this would give us an error while we download data from Y-Finance so we have to actually replace all dots with a dash And that would do the job.

Next step would be to, you know, just grab list with all the stocks.

So yeah, we can just use symbol.unique to list.

We'll assign that to object called symbols list.

And yeah, that would be our symbols list of all S&P 500 stocks.

As we talked in the beginning this list of stocks is not survivorship bias free so you need to know that it's a quite some limitation.

Alright so we would like to download the data up to a few days ago.

Let's define an object which is end date and we'll use 2023 September 27th and the start date would be...

It would be exactly 8 years ago so we can just use the end date and we'll subtract 8 years out of it.

We'll convert to date time the end date and then we'll just subtract pandas date offset and then supply 365 times 8.

All right so now we have the end date which is 27th of September which is a string and the start date would be a timestamp but that's alright because we are going to download data from yfinance and this function we'll use from YFinance download function which takes tickers as an argument so that would be our symbols list then start which would be our start date and end which would be our end date and now this would download all the S&P 500 constituents if we run it yeah it would take some time.

All right so we've downloaded all the data for the S&P 500 stocks.

So the next step would be to actually, yeah, we'll comment out everything and have this object printed out.

As you can see up here, we have the column and then we have like a multi index column.

So first we have the adjusted close, and then we have the adjusted close for each stock.

That's not really convenient to work with the whole data frame.

At the moment we have 2012 rows and 3018 columns that's really inefficient.

How we can overcome that we will just use the stack method which now creates a multi index.

The first level would be the date and then for each date we have the corresponding 500 stocks adjusted close, close, high, low so that's much more convenient we have 6 columns and almost 1 million rows so we'll have to change the date frame to be stacked or actually we can move this method right up here after the download directly and yeah now we have our date frame stacked.

Next step would be I would always when I have a multi index I would always want to have labels on both of the index labels.

So we'll say index.names and we'll assign the new names.

So date and ticker, that will be our new multi-index names.

Next step would be to fix the columns a little bit.

I would like to fix the column names to be not as titles but with lower letters.

So that would be df.columns, df.columns.string lower and that's pretty much it with the downloading and fixing data a little bit before we move to the next step which would be to start calculating technical indicators and features of all those 503 stocks.

Okay so in the second step we can start calculating the features and technical indicators for each stock.

We are going to calculate the Garmin Class Volatility, RSI, Bollinger Bands, ATR, MACD and Dollar Volume for each stock.

Let's first start with the Garmin Class Volatility.

It is a volatility measure usually used in forex trading, but it works for stocks as well.

It is an approximation to measure the intraday volatility of a given asset.

We define a new column called Garmin class Vaux and it will be the following.

So log from the high minus log from the low.

So that will be from the low this whole thing is squared and then it is divided by 2 then from this we subtract 2 times log 2 minus 1 multiplied by subtraction of log from the adjusted close yeah minus log from the open gain the whole thing is squared then we just close another bracket and that should be it.

That should be the Garmin Class Volatility.

Yes, so we now have calculated the Garmin Class Volatility for each stock.

We don't need to do any fancy calculations for this 1.

So how do we calculate the RSI on each stock?

We are going to group by on the ticker level.

So level 1, the multi-index has level 0 and level 1, which is the ticker.

Level 0 is the date, level 1 is the ticker.

Then we are selecting the column which would be adjusted close and apply the transform method.

And within the transform method we'll just apply a lambda function which would be...

So now we have grouped by on each ticker and what do we want to do?

To calculate RSI we are going to use the pandas underscore TA package which is the package to calculate pretty much all of the needed technical indicators.

So we use the pandas underscore TA package and from the pandas underscore we use the RSI function.

The RSI function we have to supply the close price which would be X and the length would be 20 and yeah if we do that you see now we have the RSI for each stock.

How we can double check our work we select Apple and then RSI and then we'll plug it.

And yeah as you can see the RSI goes up and down up and down so we have worked correctly.

The next indicator we would like to calculate is Bollinger Bands and actually we would want to have the lower band, the middle band and the upper band but there is 1 specification for each indicator from now on we would like to normalize and scale the indicator itself.

So for the Bollinger bands we will supply the log from the close price.

First the function we are going to use is from PandaCA B-bands and it is taking the close just for presentational purposes.

We'll supply a upper adjusted close and the length will be 20.

When you run this function, it returns 5 columns.

The second 1 is the middle band and the third 1 is the upper band.

We have to take this into account and what we are going to do is pretty much, you know, define a new column bb low, Bollinger Bands low and we'll use the the same idea as for the RSI.

We are going to group by each ticker, select the adjusted close column and then we'll use the transform method lambda function and in the lambda function we'll say pandas ta bbounds close would be equal to X actually it will be equal to log of X so MP log 1P and then the length will be again 20 when we run that it will return those 5 columns what we actually want to do is assign to be below the first column which we know it's the lower Bollinger band.

We can say I lock all the rows and the first column.

We can repeat the same operation for the mid band and for the upper band.

Okay but this happens when I forget something and I think I have forgot the second curly bracket.

Yeah guys, after you return and calculate something you can just select all the code you've used and comment it out.

Okay we forgot to change the names that will be BB mid and first index that would be BB high the second index.

Okay Let's run that again and see what we get.

Awesome, now we have the lower Bollinger band, middle Bollinger band and upper Bollinger band and we have the data scaled and normalized.

Next step is to calculate the ATR for each stock.

However, the ATR function needs 3 inputs, so 3 columns, not only 1 column.

And when we use transform method in pandas, it is actually working when you select only 1 column.

It would not work if you have 3 columns as an input.

So we would have to use another approach, more specifically, that would be a group by apply.

And to do that, we need to define our own custom function to calculate the ATR.

See all freeCodeCamp.org transcripts on Youtube

Algorithmic Trading – Machine Learning & Quant Strategies Course with Python