Running the webapp2 framework on Heroku

What is Heroku?

Heroku is a cloud platform that helps in developing and deploying apps written in several programming languages. What I like in Heroku is that it creates a Virtual Environment for every app you publish. So, you can have your own packages and frameworks from the language of your choice. 

What is webapp2?

webapp2 is a lightweight web development framework for Python by Google. It is one of the easiest frameworks to work with in Python. In fact, I started by journey of web development by learning this framework via the Google App Engine. 

Lets begin!

1) Set up Heroku on your system

I am not going to write much about this, as Heroku has given an awesome Getting Started tutorial for Python here. Follow that link and you will get the basics of the platform and be up and running with a Hello World application in Heroku in about 15 minutes. Yes, just 15 minutes. The tutorial will also talk about something called a Procfile, its a very important part of this project so, just have an idea about what it exactly does. I highly recommend you to follow that tutorial now, if you are new to Heroku. This tutorial follows a unix shell command line interface. 

2) Now, comes the fun part

By this time, you must have installed the Heroku tool belt. 

1. Create a new folder called hellowebapp. This will be the project folder. All files must reside here.
2. Now login to heroku.

   $ heroku login

3. cd to the project directory

   $ cd hellowebapp

4. Create a virtual environment. If you don't know what this is, follow this link.

   $ virtualenv venv

5. Activate the virtual environment

$ source venv/bin/activate

6. Install WebOp, Paste and webapp2 using python's setup tools. We are going to use pip for this tutorial.

$ pip install WebOp
$ pip install Paste
$ pip install webapp2

7. Now, create a python file which defines your app. For the sake of the tutorial, we are going to run a simple hello world! example.:

   import webapp2
   
   class HelloWebapp2(webapp2.RequestHandler):
       def get(self):
           self.response.write('Hello, webapp2!')
   
   app = webapp2.WSGIApplication([
       ('/', HelloWebapp2),
   ], debug=True)
   
   def main():
       from paste import httpserver
       httpserver.serve(app, host='127.0.0.1', port='8080')
   
   if __name__ == '__main__':
       main()
   

8. Create a file called Procfile.txt

   touch Procfile.txt
   nano Procfile.txt
   

9. Add the following contents to that file and Save it

   web: gunicorn hello:app --log-file=-

10. Test your app, locally. After the execution of this command, the app will mostly run at http://localhost:5000

    $ foreman start

11. Copy the list of packages to a file called requirements.txt so that when you deploy, the web server installs the packages automatically. The pip freeze command will come to help.

    $ pip freeze > requirements.txt

12. Create a git repository and save changes

$ git init
$ git add .
$ git commit -m "First!"

13. Create a Heroku app

    $ heroku create

14. Push contents to cloud

    $ git push heroku master

15. Lets just run one dyano of the app

     heroku ps:scale web=1 

Ta-da! Your webapp2 powered app is now running on heroku! Now start exploring the framework and continue with the work. Any problems? Just comment below. I will reply asap. If you have problems in setting up Heroku, you can ask that too.

Happy web developing!

Installing and Running Apache Pig on Hadoop 2.x versions

Pig is a high-level platform for creating MapReduce programs used with Hadoop. The language for this platform is called Pig Latin. Pig Latin abstracts the programming from the Java MapReduce idiom into a notation which makes MapReduce programming high level, similar to that of SQL for RDBMS systems. Pig Latin can be extended using UDF (User Defined Functions) which the user can write in Java, Python, JavaScript, Ruby or Groovy and then call directly from the language.

Installing Pig is just as simple. What you will have to do is:

1. Download the desired pig distribution from any one of the Apache Mirrors. It is best if you choose the latest version of Pig. Also, download the file whose name is like pig-0.12.1.tar.gz, where "0.12.1"is the version number. 
2. Extract Pig to a desired directory.
1.  Simple way is to copy the tar.gz to the root directory where you want your installation to reside.
2. Now, execute the following code in Linux bash at the directory
   

   tar -xzf pig-0.12.1.tar.gz

3. You will have the installation ready.
3. Editing Path
1. To access the pig installation easily and run your scripts from anywhere, make sure to add the pig's bin in your path.
2. To do this, open the file /home/user/.bashrc in your favorite editor and copy the following line at the end of the file. 
   

   export PATH=/<my path to pig>/pig-n.n.n/bin:$PATH
   

3. After doing all this, your Pig installation is ready for further configuration.

You might get the following error when running Pig scripts with Apache Hadoop 2.x. Here is a highlight of the error:

Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Now, that's a problem. Don't worry, solving it is unbelievably simple. This is what you will have to do:

• cd  to your pig installation directory. Yes, inside the Pig directory.
• And run this code:
  

  ant clean jar-withouthadoop -Dhadoopversion=23
  

After that, try running your Pig script again. You will find that everything is alright now.
Any problems working this around or have any suggestions? Just comment it below :)