Tuesday 24 August 2010

COM / DCOM In Python : The comtypes library

There are all kinds of crazy reasons you might want to use COM in Python. There are two main libraries for doing so. If you can use pywin32, then do- its high level, Pythonic and relatively functional. If you want to instantiate Word, or Excel, call a few methods, pass some simple data in, get some simple data back, fill your boots:


For more hardcore COM stuff you need to use the much lower level and partly undocumented comtypes library. The following bullets define "hardcore":
  • Subscribing to a COM event notification sink.
  • Building your own COM server (are you mad?)
  • Sending and receiving more complex datatypes.
It was my strong desire to automate certain tasks my group is responsible for by creating a DCOM client to hook into our management agents on the several hundred servers in our datacenter. This required an event sink.

I don't want to be unfair to comtypes, it is a great library, and when it works is very satisfying and robust. The main problems are the gaps in documentation. So without further ado, here's my pointers on using comtypes.

Download it and give it a try. Python 3.0 is unsupported afaik.
Do read the existing documentation. Carefully, what exists is of good quality.
Use the mailing list archive. Seriously, its the only way forward.
Have the IDL handy. The IDL is the file which defines the COM interfaces for your object, which the TypeLib is generated from. Most likely, if you need to use comtypes this file will be available to you, and will be a vital reference.

Turn on logging. If you're having issues you can turn on comtypes logging.


Read the source of comtypes. Once nice thing about Python is the source code is very readable. There are some useful comments in there also. On my machine the source can be found:
  • C:\Python26\Lib\site-packages\comtypes
Files which I found of interest were __init__.py, automation.py (types) and client._events.py (for events stuff).

Understand the type coercion. Much of the work that comtypes does for you is in the type coercion. If you get stuck wondering why your specific type of VARIANT is not getting coerced quite how you expected then  take a look in automation.py, and read the following Mailing list post (which should really be in the docs):
Keep your Sink in scope. This may be because I'm an idiot, but I was declaring my "advise" object in a constructor and then not keeping it. Make sure your advise object is still in scope when you call PumpEvents().

Understand the domain. Chances are you are going to have to learn about the bizarre world of C++ and COM. Frankly COM could only ever have seemed to make sense to a C++ programmer. Make sure you at least have a hazy notion of the following concepts, ideally in their C++ sense:
  • Interface
  • VARIANT
  • Pointer
  • Structure
  • Array
Also, realize why COM came about. Back in the day Windows had "dll hell" so Microsoft wanted a "code registry" which allowed for multiple versions and implementations, rather than just a system folder full of DLLs. Java was on the horizon with the idea of the "component model" and "code reuse", and Microsoft needed to compete. RAD languages such as Delphi were getting big too - so Visual Basic compatibility was another talking point. Server side Software Engineering problems which Windows Server was being utilized to solve were getting larger than a single machine: so it made sense to whack a bit of RPC in the mix (DCOM).

Built on ctypes. Be mindful the comtypes is build upon ctypes, especially when you get into type based confusion the docs there can be very enlightening.

Friday 6 August 2010

Working with PayPal Web Payments Pro

For a web based store, to process credit card transactions is a requirement. PayPal offers some simpler integrations than fully blown gateways such as Authorise.net. My plan was to "ramp up" using PayPal to get a working flow, then migrate to a "proper gateway" later on if it seemed justified.

Getting PayPal up and running is simpler than it seems, however I found there is something "bewildering" about the PayPal website, the range of options they offer and their sandbox. This is to share the things I have learnt.

Inappropriate Options

Mass Payments is sending money to a large number of people. Not useful for me. Website Payments Standard offers lots of "clever" buttons which you can embed on a static site. Forget about it. Mobile Checkout is engineered for WAP. Not interested. Payflow Pro has PayPal act as an intermediary to a third party gateway. Can't see the point in this. Website Payment Pro Payflow edition is just a variant on Payflow, so again, not interested.

Website Payments Pro (Express Checkout and Direct Payment)

Once I focussed my mind on Website Payments Pro, things slotted into place. WPP consists of Express Checkout - where you dump your customer at PayPal and they use their PayPal account, and Direct Payment where you grab customer's credit card data your end, and post it to PayPal in the background. Direct Payment does not require a PayPal account - however it should be obvious that SSL is a requirement, as you are grabbing sensitive info your end.

Direct Payments is the real win here effectively its a very simple to use payment gateway at a reasonable cost from PayPal. However - the WPP offering is the combination of them both - Express Checkout and Direct Payments - so if you implement Direct Payments you are obliged to offer Express Checkout too - in order to redirect people into creating PayPal accounts I would warrant. This is not too much sweat however.

1. Creating a PayPal developer account.

First step is to create a PayPal developer account, this is not the same thing as the account you will use to access the API - this is the account you need to then create test API accounts. So this is a first step:
2. Create Sandbox account to test with.

Once you are logged into the PayPal sandbox "center" then you can create an account within the Sandbox to test with. There is a macro for creating preconfigured types of accounts- effectively you want the test version of a merchant account with WPP enabled. Select "add account" and then add a WPP test account. It will generate some rubbish email address based on what you enter. Emails sent to this address in the course of testing (i.e. notification of purchases etc) will go to an integrated webmail section of the Sandbox site. Don't forget the password you use to create the sandbox merchant account! It is different from the API password and you will need it to view the status of transactions during testing.

At this point also treat yourself by setting up an additional account to act as the customer when testing express payments.

Once you have created your Sandbox merchant WPP account, you can then browse the details of this account on the "API Credentials" menu item. You should have the following details:
  • API_USER_NAME: This is just a test Merchant username for API calls.
  • API_PASSWORD: The password.
  • API_SIGNATURE: This is the additional signature for accessing the API.
However, with these three items you should be set to access the API. I'll be doing so using Python.

3. Hit the NVP API webservice and do stuff.

At this stage its worth digressing on the sorts of flows which will need to be implemented. Step -1 is that your marketing will work, the customer will come to your site, feel excited / relieved to have found something they want to buy, and will use whatever system of button pressing you have implemented to add the item to their shopping cart. The following account begins at the point where they click "checkout".

Express Checkout:
  1. You offer the customer the option to enter credit card details on your site (Direct Payments) or use Express Checkout. They choose the latter.
  2. You POST using the NVP API to the SetExpressPayment method. This POST includes your API key, the amount of the transaction plus return and cancel URLs for PayPal to return your customer to.
  3. You receive an "express checkout token" in response. You construct a URL based off a static PayPal URI plus this token, and redirect your customer to it.
  4. The customer completes their purchase in PayPal by either entering their PayPal account details or creating a new PayPal account.
  5. PayPal will then redirect the customer back to your site, appending URL parameters containing the Express Checkout token and the Payer ID, which identifies the customer.
  6. You then call DoExpressCheckoutPayment method using the NVP API passing in the "express checkout token", the Payer ID you got back in step 5 and the same amount you passed along in the first instance. PayPal sends you a confirmation as a reply in NVP.
  7. The customer gets a confirmation email from PayPal, and the test merchant account has the transaction posted into it. You can check this from the Sandbox "center" site "Test Email" section.
So here's a sample script for this which can be run from the command line / IDLE etc. Note you need to copy paste the payer ID from query URL in the browser after you get back from PayPal into the script.


The above is heavily based on the example here
So that's Express Checkout - given this post has turned into a complete monster I'm going to leave Direct Payments for another day and another post. Hope someone finds this post useful!