Spark streaming reduceByKeyAndWindow unstable application

I wanted to run a job which runs 24×7 and which reports if certain keywords occur more than a N times in the stream. Spark streaming looked a ideal candidate for this task. Spark has a reduceByKeyAndWindow function which was exactly what I was looking for. I decided to use a window length of 1 minute and […]

Motorcycle review: Bajaj Avenger 2013

I bought Avenger in July 2013. Specification can be read at http://www.bajajauto.com/avenger/. I have toured extensively in Karnataka and Leh, so I think this review will definitely help you make up your mind. General: Engine: The 220 cc engine gets you off from 0-60 pretty quickly. This is a cruiser so couple of seconds does not […]

Tips for planning your motorcyle trip to Leh

Planning: Read and research. Start here: http://www.bcmtouring.com/travel-gui…-travel-guide/ http://www.team-bhp.com/forum/route-…ate-guide.html See lot of photos, will keep you excited 🙂 You can ride solo, however I would highly recommend against it. Form a group on Bangalore Bikers or BMC Touring. Ride with experienced bikers. Newbies will find it daunting to ride everyday. I had a guy in my group […]

adding a non standard library path to linux library path

Sometimes when you compile some libraries like opencv, they install the compiled shared libraries in non standard path /usr/local/opencv/lib instead of /usr/lib or /usr/local/lib.  So when you try to compile your code against these libraries, gcc will not be able to locate them or when you are using python bindings, python wont be able to locate them. […]

Increasing max open files limit on Ubuntu

To ensure good server performance, the total number of client connections, database files, and log files must not exceed the maximum file descriptor limit on the operating system (ulimit -n). Linux systems limit the number of file descriptors that any one process may open to 1024 per process. Edit /etc/security/limits.conf and add these lines to the […]

WFH

After a brief stint at Mountain View, I came back to Bangalore and have been working from home since then. Working from home (rather from a remote location for me, since rest of my team is on the other half of the world) has its own advantages and disadvantages. I am listing down some of […]

My desk

This is my desk. So what do we have here: Lenovo Thinkpad T530 Dell H2312HM Monitor Microsoft Ergonomic 4000 Keyboard 3M Ergonomic mouse Apple Mini Displayport to DVi Adapter Creative 2.1 Speaker system (not visible in the pic) All the products are from different vendors. I had not noticed this until I took this pic.

uptime

$uptime 16:00:45 up 5 days, 8:40, 5 users, load average: 0.52, 0.45, 0.28 That’s my Thinkpad 🙂