Paper on "Application of deep and reinforcement learning to boundary control problems"

Recently, I wrote a paper based on my master thesis project, which I had attempted to submit to AAAI conference. However, the attempt was futile as the reviewers did not really like it. It was quite interesting to see that at least some of the comments by the reviewers are contradictory with each other; it looks like I am not the only one who is lazy to read. Anyway, I decided to make it available for the public via arXiv and viXra. The following are the links to the same. Originally, I had only planned on submitting this to arXiv. When I checked the submission portal a couple of hours after the scheduled publishing time, I saw that the article was put "On Hold" from being published. I searched for the reasons for the same, and I read in a few places that sometimes arXiv takes a lot of time to publish them once put on hold, and sometimes they just don't publish them at all. Therefore, I decided to submit it

Phone as Trackpad for your Computer

I am pretty sure that I am not the only one who crave for a mouse when I don't have one and crave for a trackpad when I don't have access to one. I have a few ideas on turning your phone into a mouse and a trackpad. Let us focus on the later in this blog post. There are two components to this experiment; the first is a server on the computer that you wish to control and the second is a web app that is opened up on your phone that will transmit coordinate data to the server. I think we'll create the server in Python and the web app in JavaScript.

Let's get started with creating the server. We'll use 'Flask' to handle API requests and 'PyAutoGUI' to control the mouse. We'll have APIs defined equivalent to cursor move and click for now.
import pyautogui
from flask import Flask, render_template
app = Flask(__name__)

# Apparently flask doesn't support negative numbers. So offsetting it.
coordinateShifter = 1000

def index():
f = open("index.html", "r")

def move(x, y):
pyautogui.move(x - coordinateShifter, y - coordinateShifter)
return 'moving'

def click():
return 'clicking'

To run the python server, simply run the following commands on your terminal.
flask run -h <your_ip_address>
You can experiment with the move and click APIs by opening the following links on your browser and observe the mouse moving and clicking.
Now, we need to create a Web App that can detect click and move using a touch screen and invoke the above URLs as API requests.

The HTML and CSS may designed to display a DIV that covers the entire screen. The following snippets respectively depict the same.
<div id="fullscreen"></div>


#fullscreen {
    background-color: #cccccc;
    text-align: center;
    position: absolute;
    top: 0;
    left: 0;
    right: 0;
    bottom: 0;

The JavaScript logic would be as follows:

 // Apparently flask doesn't support negative numbers. So offsetting it.

const coordinateShifter = 1000;

const elem = document.getElementById("fullscreen");

var prevX, prevY;

elem.addEventListener('mousedown', function (e) {


    var url = window.location.href + 'click';

    var xmlHttp = new XMLHttpRequest();"GET", url, false);


}, false);

elem.addEventListener('touchstart', function (e) {

    e = e.originalEvent || e;


}, false);

elem.addEventListener('touchmove', function (e) {


    if (e.targetTouches.length === 1) {


        if (typeof prevX === 'undefined') {

            prevX = e.targetTouches[0].pageX;

            prevY = e.targetTouches[0].pageY;

        } else {

            var diffX = e.targetTouches[0].pageX - prevX;

            var diffY = e.targetTouches[0].pageY - prevY;

            prevX = e.targetTouches[0].pageX;

            prevY = e.targetTouches[0].pageY;

            var url = window.location.href + 'move/' + parseInt(coordinateShifter + diffX) + '/' + parseInt(coordinateShifter + diffY);

            var xmlHttp = new XMLHttpRequest();

  "GET", url, false);





}, false);

elem.addEventListener('touchend', function (e) {


    prevX = undefined;

    prevY = undefined;

}, false);

Open up the web app on your phone using the URL http://<your_ip_address>:5000 and start controlling your mouse. You can observe that the controls are not very smooth, but neverthless, it works. The following is a video recording wherein I try to control the mouse using my phone. I did not have another camera to take the entire video, therefore I am just sharing the screencast. Note that only cursor movement and clicking have been implemented. You could also attempt to create a click and drag mechanism or multitouch scrolling as well.

Let me also attach the screenshot of the web app. Plain and simple, huh? If you are worried about the battery consumption, feel free to change the colout to black.

I hope this was inspiring. Feel free to modify the code and create something awesome.


Popular posts from this blog

Bought a new domain -

Started a blog under HexHoot

Created a video to help people get started with HexHoot