Skip to main content

Extracting Stock Price Data Trend from Google Search to Train LSTM Network

I am not very good with designing my own neural networks. I have attempted to create a few in the past, some worked out fine for proving certain points that I wanted to make, but whenever I tried to make something to participate in a competition, things were not very well. In this post, I do not intent on creating an LSTM based neural network to predict stock prices, rather simply use Google as a tool to extract the stock trends from the graphs.

To get started, search for "Amazon stock price" on google. You would be able to see a pretty nice graph. On right clicking on the graph and clicking on Inspect and reading through Elements in the Developer tools, it can be observed that the graph is rendered using Scalable Vector Graphics or SVG. This is an XML-based vector image format and the data required to create such a graphic would be available in a format that we'll be able to read. I also observed that the required SVG image has a class name uch-psvg and there is only one element with that class name.

Let us start observing the data inside the SVG image. It can be seen that identical data is being stored in the first two path tags inside the SVG. This represents the data trend. To train an LSTM, you don't necessarily need the data with right numbers; you just need the data with the right trend. Let us extract this data into variables named xValues and yValues.

svg = document.getElementsByClassName('uch-psvg')[0];
pathStr = svg.getElementsByTagName('path')[1].outerHTML;
valueStr = pathStr.split('d="M ')[1].split('"')[0];
valueStrSplit = valueStr.split(" L ");
var xValues = [];
var yValues = [];
for(var i = 0; i < valueStrSplit.length; i++){
    xy = valueStrSplit[i].split(" ");
    xValues.push(parseFloat(xy[0]));
    yValues.push(-1 * parseFloat(xy[1]));
}



It can be observed that xValues are just equidistant values and from analyzing a trend perspective, it would not provide a lot of information. Let us ignore that.

The reason for adding a -1 multiplier to yValues is because in browsers, while rendering, the coordinate axes start from the top left corner of your screen and positive Y-direction is downward and hence SVG would have values adjusted accordingly. We are only interested in the right trend and hence to flip it, we simply have to add a negative sign.

You can use this method to create a training data with trends from different stock prices and create a huge training data and train an LSTM network.

Comments

Popular posts from this blog

How to Create your own Ad Blocker Chrome Extension

Not very long ago, I read an article stating that Google banned advertisement blockers from its stores. There are several websites which does not let you in if you have an Ad blocker extension installed on your browser. I have not investigated on how these websites detect that, nor do I know whether our own version of advertisement blocker would be immune to that. But, isn't it fun to build something like this on our own? As the first step, I found a random website that has a Google Ad. I right clicked on the Google Ad and clicked on Inspect button. As I moved up the Elements inspector, I found that the id of the advertisement starts with "google_ads_". Therefore, I opened the console and typed in the following command: div = document.querySelector('[id^="google_ads_"]') This selected the div  that contained the advertisement. I went ahead and typed in the following command that would remove the div from the page. div.parentElement.remove...

Turning your phone into a virtual-joystick

Update: I kept working on this and I have released it as a package for Windows, Linux and macOS. Check it out: https://github.com/zenineasa/joystick/releases/tag/v1.0.0 -- During the days when I was pursying my master's programme, my friends and I used to occasionally go to a classroom in the university, turn on a projector, connect devices like Nintento Switch or Gaming computers and loads of joysticks, and play different simple multiplayer games; MarioKart was my favourite. From time-to-time, when I get together with people, I ponder that it would be a good idea if I bought such devices. Indeed, I do have a laptop, which could easily run such games; SuperTuxKart is similar enough to MarioKart and it can run on Linux, Windows and Mac. However, I do not have joysticks with me at the moment. Therefore, I think it would be a good idea if I simply worked on a project that would enable using our phones as joysticks. From a high-level, the plan is to host APIs on a NodeJS server that wo...

How to remove Blogger (or any free blogging platform) credits from the footer

I am not sure if removing the footer credits is legal or not, but I just wanted to try doing it. As I searched the internet, I found several pages describing how it could be done by making some adjustments in CSS. However, I could not find an article which uses simple JavaScript commands to do the same. So, I decided the write this. As you can see now, my footer does have Powered By Blogger credit signature. I am intending on keeping it that way until I am sure that removing it would not get me into any legal issues. I am just writing this article for the educational purposes. Right click on the Powered By Blogger  button and click on Inspect to have Chrome Developer tools loaded with the div that holds the credit content. You can see that the element should have a content similar to the following. Notice that the div is having a class value blogger . We can use this to our advantage. <div class="blogger"> <a href="https://www.blogger.com...