Showing posts with label python. Show all posts
Showing posts with label python. Show all posts

Saturday, 31 March 2012

Installing Python nicely on Windows box

At the time of writing despite x64 Windows use Python 2.7 (not 3.0) and x86 (not x64).
The Windows virtualenv wrapper notes are here: https://github.com/davidmarble/virtualenvwrapper-win

At this point ready to use VirtualEnv, along with requirements files etc. I suppose the process might look something like this:

  • make project folder new_proj
  • cd into new_proj
  • make requirements.txt
  • edit requirements.txt add say "flask"
  • mkvirtualenv new_proj_env
  • pip install -r requirements.txt
Additionally the postgres database driver might be required. This is in combination with a local installation of Postgres.

Wednesday, 12 January 2011

Install latest Python on Centos

This script will download, compile and install a newer version of Python on CENTOS / RedHat 5.* in a way which respects the underlying Python 2.4 used by Yum etc.

Based on http://www.venkysblog.com/install-python264-modwsgi-and-django-on-cento and http://binarysushi.com/blog/2009/aug/19/CentOS-5-3-python-2-5-virtualevn-mod-wsgi-and-mod-rpaf/.

Sunday, 2 January 2011

Unicode

Seems like a good article, this has always seemed a little mysterious.
I especially found the part about http-equiv content type useful. It had always bothered me how you could specify the character encoding after the start of the file. As it turns out it is a compromise and I am not the only person who had wondered about it.

Tuesday, 30 November 2010

Tuesday, 24 August 2010

COM / DCOM In Python : The comtypes library

There are all kinds of crazy reasons you might want to use COM in Python. There are two main libraries for doing so. If you can use pywin32, then do- its high level, Pythonic and relatively functional. If you want to instantiate Word, or Excel, call a few methods, pass some simple data in, get some simple data back, fill your boots:


For more hardcore COM stuff you need to use the much lower level and partly undocumented comtypes library. The following bullets define "hardcore":
  • Subscribing to a COM event notification sink.
  • Building your own COM server (are you mad?)
  • Sending and receiving more complex datatypes.
It was my strong desire to automate certain tasks my group is responsible for by creating a DCOM client to hook into our management agents on the several hundred servers in our datacenter. This required an event sink.

I don't want to be unfair to comtypes, it is a great library, and when it works is very satisfying and robust. The main problems are the gaps in documentation. So without further ado, here's my pointers on using comtypes.

Download it and give it a try. Python 3.0 is unsupported afaik.
Do read the existing documentation. Carefully, what exists is of good quality.
Use the mailing list archive. Seriously, its the only way forward.
Have the IDL handy. The IDL is the file which defines the COM interfaces for your object, which the TypeLib is generated from. Most likely, if you need to use comtypes this file will be available to you, and will be a vital reference.

Turn on logging. If you're having issues you can turn on comtypes logging.


Read the source of comtypes. Once nice thing about Python is the source code is very readable. There are some useful comments in there also. On my machine the source can be found:
  • C:\Python26\Lib\site-packages\comtypes
Files which I found of interest were __init__.py, automation.py (types) and client._events.py (for events stuff).

Understand the type coercion. Much of the work that comtypes does for you is in the type coercion. If you get stuck wondering why your specific type of VARIANT is not getting coerced quite how you expected then  take a look in automation.py, and read the following Mailing list post (which should really be in the docs):
Keep your Sink in scope. This may be because I'm an idiot, but I was declaring my "advise" object in a constructor and then not keeping it. Make sure your advise object is still in scope when you call PumpEvents().

Understand the domain. Chances are you are going to have to learn about the bizarre world of C++ and COM. Frankly COM could only ever have seemed to make sense to a C++ programmer. Make sure you at least have a hazy notion of the following concepts, ideally in their C++ sense:
  • Interface
  • VARIANT
  • Pointer
  • Structure
  • Array
Also, realize why COM came about. Back in the day Windows had "dll hell" so Microsoft wanted a "code registry" which allowed for multiple versions and implementations, rather than just a system folder full of DLLs. Java was on the horizon with the idea of the "component model" and "code reuse", and Microsoft needed to compete. RAD languages such as Delphi were getting big too - so Visual Basic compatibility was another talking point. Server side Software Engineering problems which Windows Server was being utilized to solve were getting larger than a single machine: so it made sense to whack a bit of RPC in the mix (DCOM).

Built on ctypes. Be mindful the comtypes is build upon ctypes, especially when you get into type based confusion the docs there can be very enlightening.

Friday, 6 August 2010

Working with PayPal Web Payments Pro

For a web based store, to process credit card transactions is a requirement. PayPal offers some simpler integrations than fully blown gateways such as Authorise.net. My plan was to "ramp up" using PayPal to get a working flow, then migrate to a "proper gateway" later on if it seemed justified.

Getting PayPal up and running is simpler than it seems, however I found there is something "bewildering" about the PayPal website, the range of options they offer and their sandbox. This is to share the things I have learnt.

Inappropriate Options

Mass Payments is sending money to a large number of people. Not useful for me. Website Payments Standard offers lots of "clever" buttons which you can embed on a static site. Forget about it. Mobile Checkout is engineered for WAP. Not interested. Payflow Pro has PayPal act as an intermediary to a third party gateway. Can't see the point in this. Website Payment Pro Payflow edition is just a variant on Payflow, so again, not interested.

Website Payments Pro (Express Checkout and Direct Payment)

Once I focussed my mind on Website Payments Pro, things slotted into place. WPP consists of Express Checkout - where you dump your customer at PayPal and they use their PayPal account, and Direct Payment where you grab customer's credit card data your end, and post it to PayPal in the background. Direct Payment does not require a PayPal account - however it should be obvious that SSL is a requirement, as you are grabbing sensitive info your end.

Direct Payments is the real win here effectively its a very simple to use payment gateway at a reasonable cost from PayPal. However - the WPP offering is the combination of them both - Express Checkout and Direct Payments - so if you implement Direct Payments you are obliged to offer Express Checkout too - in order to redirect people into creating PayPal accounts I would warrant. This is not too much sweat however.

1. Creating a PayPal developer account.

First step is to create a PayPal developer account, this is not the same thing as the account you will use to access the API - this is the account you need to then create test API accounts. So this is a first step:
2. Create Sandbox account to test with.

Once you are logged into the PayPal sandbox "center" then you can create an account within the Sandbox to test with. There is a macro for creating preconfigured types of accounts- effectively you want the test version of a merchant account with WPP enabled. Select "add account" and then add a WPP test account. It will generate some rubbish email address based on what you enter. Emails sent to this address in the course of testing (i.e. notification of purchases etc) will go to an integrated webmail section of the Sandbox site. Don't forget the password you use to create the sandbox merchant account! It is different from the API password and you will need it to view the status of transactions during testing.

At this point also treat yourself by setting up an additional account to act as the customer when testing express payments.

Once you have created your Sandbox merchant WPP account, you can then browse the details of this account on the "API Credentials" menu item. You should have the following details:
  • API_USER_NAME: This is just a test Merchant username for API calls.
  • API_PASSWORD: The password.
  • API_SIGNATURE: This is the additional signature for accessing the API.
However, with these three items you should be set to access the API. I'll be doing so using Python.

3. Hit the NVP API webservice and do stuff.

At this stage its worth digressing on the sorts of flows which will need to be implemented. Step -1 is that your marketing will work, the customer will come to your site, feel excited / relieved to have found something they want to buy, and will use whatever system of button pressing you have implemented to add the item to their shopping cart. The following account begins at the point where they click "checkout".

Express Checkout:
  1. You offer the customer the option to enter credit card details on your site (Direct Payments) or use Express Checkout. They choose the latter.
  2. You POST using the NVP API to the SetExpressPayment method. This POST includes your API key, the amount of the transaction plus return and cancel URLs for PayPal to return your customer to.
  3. You receive an "express checkout token" in response. You construct a URL based off a static PayPal URI plus this token, and redirect your customer to it.
  4. The customer completes their purchase in PayPal by either entering their PayPal account details or creating a new PayPal account.
  5. PayPal will then redirect the customer back to your site, appending URL parameters containing the Express Checkout token and the Payer ID, which identifies the customer.
  6. You then call DoExpressCheckoutPayment method using the NVP API passing in the "express checkout token", the Payer ID you got back in step 5 and the same amount you passed along in the first instance. PayPal sends you a confirmation as a reply in NVP.
  7. The customer gets a confirmation email from PayPal, and the test merchant account has the transaction posted into it. You can check this from the Sandbox "center" site "Test Email" section.
So here's a sample script for this which can be run from the command line / IDLE etc. Note you need to copy paste the payer ID from query URL in the browser after you get back from PayPal into the script.


The above is heavily based on the example here
So that's Express Checkout - given this post has turned into a complete monster I'm going to leave Direct Payments for another day and another post. Hope someone finds this post useful!

Saturday, 31 July 2010

Production Flask app on CENTOS and Apache via mod_wsgi

Flask is a Python web framework which embraces simplicity and seems to work. It is based on Werkzeug so it is a WSGI application.

I've had some luck in the past using mod_wsgi to run Django apps, so I suspect getting a Flask app to run will be similar.

Using Google, I found the following in the Werkzeug Docs, which I reviewed for the following steps:

1. Install python2.7, mod_wsgi and Flask on production server.

Using Centos 5.4 (unbranded RHEL) comes with with python 2.4 and apache 2.2. Centos (and RHEL) use Python extensively under the hood (for Yum, among other things). I want to use a later version of Python, but one may not simply "upgrade" the version of Python- without breaking the OS - you need to install an additional version. I followed the steps in this guide, under the head "Install Python":
I chose to use Python 2.7 as its the last stable release of Python 2, and I don't intend going Python 3.* for a while. I hope I don't regret that. I needed to change a few things in obvious places as the guide covers Python 2.6.


Next I carried out the steps labelled "Install Setuptools" as this is necessary.

Next I carried out the steps labelled "Install mod_wsgi", restarted httpd and it started!

To install Flask I ran SetupTools explicitly from the location inside of /opt/python2.7/ to install 'pip' into the 2.7 Python distribution. Next I added an alias to pip to the ~/.bash_profile in the same way this was done for the python binary in the VenkysBlog page. Finally I used the source command to load the alias and ran:

pip install Flask

2. Create an appropriate wsgi file.


WSGI uses a file with the extension .wsgi as the interface between the wsgi container (in this case Apache / mod_wsgi) and the app. Flask has excellent documentation:

Steps are as follows:

  1. Place Flask application in a subfolder of the webroot (for testing I just used the "Flask is Fun" example: http://flask.pocoo.org/).
  2. Make a file called 'yourapp.wsgi' in the same structure - this is written in Python syntax - here you need to import your app and bind it to the name 'application'. mod_wsgi will then call this object in the appropriate way.

3. Edit apache configuration to invoke wsgi application.


Finally some directives need adding to httpd.conf. The LoadModule statement was added as part of step 1, however the wsgi file need to be referenced vs. the appropriate virtual root.



Great so everything worked! Wasn't that easy?

Tuesday, 1 June 2010

Getting VirtualEnv going

Everywhere I read the benefits of virtualenv, but each time I try it I get fed up as it seems a lot to get setup. Here's to trying again:.

yum - y python-setuptools
easy_install pip
pip install virtualenv

Friday, 2 April 2010

Using mod_wsgi to support Django development via Apache

I'm building a Django shop which integrates PHP components in the form of a blog (Wordpress) and needs testing with SSL. As discussed in the last blog entry- that means I can't well test holistically using only the Django dev server. On production I am using mod_python at the moment, which is inappropriate for development as I would need to restart the server each time I make a change.

I'm looking to follow a recipe, using mod_wsgi for two purposes:
  • To have a dev server running via apache, to integrate PHP and SSL components, as well as having "hot code replacement" to support development.
  • To evaluate mod_wsgi on the basis of the claim: "for hosting WSGI applications in conjunction with Apache it has a lower memory overhead and performs better than mod_python".

Step 1 will be to deploying using mod_wsgi on my dev laptop per the mod_wsgi docs.
  • sudo apt-get install libapache2-mod-wsgi [ubuntu]
  • restart apache
Step 2 is a standard integration with Apache. Per http://code.google.com/p/modwsgi/wiki/IntegrationWithDjango
  • create appropriate .wsgi file
  • edit apache.conf with directive (see below)
Step 3 is getting it to then run in daemon mode. This is a little like what mod_jk or ruby passenger / mod_rails does. It brokers requests off to a separate sub process which forks and brokers off to individual Python Django scripts.

Step 4, finally per: http://blog.dscpl.com.au/2008/12/using-modwsgi-when-developing-django.html, is to add a monitor.py (the source of which is here) and reference it in the .wsgi file.

Here are example apache / httpd.conf:


And dev.wsgi (based of putting a monitor.py in the root of the django app):

Thursday, 7 January 2010

A configuration file with an encrypted password in Python

I need to set configuration by environment in quite a few Python utils. I'd like to create an object with a filename as a constructor variable, and then get a dict of the values. One issue is the password -  better not to keep it in plain text. So I need to make them "encrypted", or a better phrase might be "non-human readable".



http://gist.github.com/273393

I got my head round the python unittest framework when writing this, and also got my head round the ConfigParser and optparse libraries.

Wednesday, 30 December 2009

Python: Determine location of Site-Packages / Dist-Packages

Nice tip from the Django tutorial page.

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

http://docs.djangoproject.com/en/dev/topics/install/