Current Python Development Environment

In my day job I program exclusively in the Windows environment.  I pushed hard to get Python into our development environment here.  I love Python, what a great language it is!  The problem with Python is the development tools, or as any Windows programmer will whine: “You can’t use Visual Studio!” (insert sobbing noises here).  Visual Studio spoils us, and when we can’t use it, we’re as sad as a MAC fanboi that just dropped his iPhone in the toilet.  I should say the problem with free Python development tools as I had no budget to try commercial tools.  I have heard that some of the commercial Python development tools, like Wing IDE, are quite good. But, after blowing the budget on Visual Studio, I was left to choose from amongst the free tools.

Eclipse is almost Visual Studio.  Eclipse is a very nice IDE.  Add the PyDev tools from Aptana and it is a great Python development environment.  It supports Jython and IronPython development as well.  This environment support many features including:

• You can configure PyLint for static analysis. 
• Source level debugging, breakpoints, the whole bit.
• You can configure and use multiple Python interpreters.
• Constantly updated and improved, (thanks Fabio!)

Grab the latest Eclipse Galileo at http://www.eclipse.org/downloads/  The PyDev home page is here http://pydev.org  Python is available here http://python.org

SharpDevelop wants to be Visual Studio.  SharpDevelop is an open source IDE for the .NET world.  It is taking on Visual Studio directly, come one, Visual Studio is like a MacBook Pro, waaayyy overpriced! SharpDevelop supports an impressive array of languages: Boo, C#, F#, ILAsm, IronPython, and VB.NET.  SharpDevelop 3.1.1 comes with IronPython 2.6 in the box.  It also gives you a drag and drop Windows Forms editor just like Visual Studio, except that you can use Python! 

Get SharpDevelop here http://www.icsharpcode.net/OpenSource/SD/Download/ and IronPython at this Codeplex download page

Unfortunately I have not had the opportunity to really use IronPython for anything real.  I have written a few toy applications with it.   I plan to spend more time with SharpDevelop and IronPython in the near future.  I have read the wonderful book “IronPython in Action” by Micheal Foord and Christian Muirhead.  My first project will be to get the cherrypy web framework working under IronPython.  Right now about half of the cherrypy test suite passes, it’s a work in progress.  But that is for another blog entry.

Useful Debugging/Reversing Web Sites and Tools

IDA Pro Disassembler and Debugger:
http://www.hex-rays.com/idapro

Olly Debugger:
http://www.ollydbg.de

Immunity Debugger (based on Olly but with bug-fixes and Python scripting!):
http://www.immunityinc.com/products-immdbg.shtml
http://forum.immunityinc.com/

PaiMei/PyDbg a reverse engineering framework:
http://paimei.googlecode.com

OpenRCE (Open Reverse Code Engineering):
http://www.openrce.org

Dump Analysis:
http://www.dumpanalysis.org

Using Breakpoints to Skip a Function in WinDbg

The following is an example of how to skip a function (even if you don't have the source) in a device driver. Look at the assembly below. This is the function start/end for a 3^rd party device driver function that I wish to fail without executing. First I let the stack frame get set up so I must also tear it down. I set the breakpoint at instruction f7ed0511 so that before this line executes, the breakpoint will go off. Then I give a command to the break point. The command 'reax = c0000001' sets the eax register to STATUS_UNSUCCESSFUL. The command 'reip = f7ed0735' sets the instruction pointer to the line 'pop edi' to begin the stack frame teardown. The final command 'g' tells the debugger to go. So each time this breakpoint is executed, the function is skipped with a failure code and the debugger continues execution.  You can even use the .echo command to display a message every time the breakpoint goes off.

1: kd> u fslx+7506
fslx+0x7506:
f7ed0506 8bff             mov edi,edi
f7ed0508 55               push ebp
f7ed0509 8bec            mov ebp,esp
f7ed050b 53               push ebx
f7ed050c 8b5d08        mov ebx,dword ptr [ebp+8]
f7ed050f 56                push esi
f7ed0510 57               push edi
f7ed0511 6846583430 push 30345846h
1: kd> u fslx+7735
fslx+0x7735:
f7ed0735 5f       pop edi
f7ed0736 5e      pop esi
f7ed0737 5b      pop ebx
f7ed0738 5d      pop ebp
f7ed0739 c21800     ret 18h

1: kd> bp f7ed0511 "reax = c0000001; reip = f7ed0735; g"
1: kd> bl
1 e f7ed0511 0001 (0001) fslx+0x7511 "reax = c0000001; reip = f7ed0735; g"

With the .echo command:
1: kd> bp f7ed0511 "reax = c0000001; reip = f7ed0735; .echo \"skipping function...\"; g"

Exceptions: Win32 Structured Exceptions, C++ Exceptions, and Compiler Options.

In a Windows program that is written in the C++ language there are two types of exceptions that can be handled. The first is the Win32 structured exception that is built into the OS. The second are the C++ exceptions that are generally classes derived from std::exception.

To handle a Win32 structured exception you would have code like so:

__try {
   // here is my code. 
} __except(EXCEPTION_EXECUTE_HANDLER) {
   //  handle the exception here 
}

To handle a C++ exception you would have code like this:

try {
   //  here is my code. 
}
catch(std::exception &e) {
   //  handle exception here, should handle specific 
   // exceptions that you expect in production code 
}
catch(...) {
   //  handle any other type of exception 
   //  i.e.  not derived from std::exception 
}

The question is: What happens when a win32 SE is thrown and you are using C++ exceptions?

The answer: It depends on your compiler options!

Project Properties->Configuration Properties->C/C++->Code Generation->:Enable C++ Exceptions:

The choices are:
No
Yes (/EHsc)
Yes with with SEH Exceptions (/EHa)

If you use the setting No, you will get compiler warning C4530 for using C++ exceptions when they are not enabled. If you are treating warnings as errors like all professional programmers do, then you fix this compiler setting before going on. Runtime behavior is similar to Yes (/EHa).

If you use the setting Yes (/EHsc) C++ exceptions will be caught in catch handlers as you would expect. Win32 Structured Exceptions WILL NOT BE CAUGHT even by a catch(...) block!

If you use the setting Yes(/EHa) C++ exceptions will be caught in catch handlers as you would expect. Win32 Structured Exceptions WILL BE CAUGHT in catch(...) handlers only.

The moral of this story is: use the Yes(/EHa) setting at all times!

With the use of the Yes(/EHa) setting, you can use the function _set_se_translator to register a function that would be responsible for translating all Win32 structured exceptions into C++ exceptions. This helps to unify the error handling framework. Now you can use strongly typed C++ exceptions throughout your program. You should never have to use a catch(...) block again!

Example:

void my_translate(unsigned int code, _EXCEPTION_POINTERS *ep)
{
   std::runtime_error re("translated win32 SE");
   throw re;
}

// ... somewhere in WinMain... 
_set_se_translator(my_translate);

In Windows each thread has its own translator function which means that you must install your translator on each thread that you create.

auto Keyword Re-purposed for C++ 0x

In the coming C++ 0x (1x ?) standard, the auto keyword has been re-purposed to behave like the var keywork in C# 3.0. Currently auto is a storage class specifier that almost no one uses. For example, the following C++ statements are equivalent:

auto int x = 5;
int x = 5;    //implicitly auto

Being that this is the default, no one uses the auto keyword. So now this use of auto has been removed from C++. It is now a note to the compiler to determine the type of a variable from the usage context. For example:

std::vector<std::string> vs;
std::string str = "foo";
vs.push_back(str);

// we all hate to type this stuff!
std::vector<std::string>::iterator iter = vs.begin();

// now we can just type this:
auto iter = vs.begin();

I can't wait until this is supported in Visual Studio, maybe version 13 in ten years or so...

Install Windows 7 from a USB Flash Drive

This is a simple process that will allow you to boot to a USB flash drive to install Windows. First we must prepare the USB flash drive. The flash drive that is selected will be completely wiped out so make sure that it does not contain any data that is important to you.

1. Run a command prompt as an Administrator

2. Start the diskpart program, type: diskpart

   1. Diskpart is a command line disk partitioning tool that accepts commands that you type.

3. Type the commands:

   1. List Disk

      1. This show a list of all disk on your computer, make a note of the disk number of your USB flash drive

   2. Select Disk <n>

      1. Where <n> is the disk number from the List disk command, MAKE SURE you enter the correct number.

   3. Clean

      1. Wipe out the disk
   4. Create partition primary

   5. Active

      1. These two commands create a primary partition and mark it as active.

   6. Format fs=fat32 quick

      1. Format the disk with the Fat32 file system, quick is optional.

   7. Assign

      1. This assigns a drive letter to the USB flash drive in Windows Explorer.
4. Insert the Windows installation disk into your DVD drive.
5. Copy the contents of the Windows Installation disk to your USB flash drive

Now you can take that flash drive over to your netbook and install Windows 7!

How to Disable Code Analysis on 3rd Party Headers

This little snippet will disable the code analysis features of Visual C++ on header files that, for whatever reason, you can not edit or fix. This does include some Windows header files (especially ATL).

#include <codeanalysis/warnings.h>

#pragma warning (push)
#pragma warning (disable: ALL_CODE_ANALYSIS_WARNINGS)

// include 3rd party headers here

#pragma warning (pop)

I know this is in MSDN, but it’s easier to search this blog than MSDN!

I have chosen Pygments for my source code formatting. It is a Python library that handles many languages. It is the best (of about 10 or so) that I have tried by far. I am very impressed.

Hello!

This blog has grown out of my own need to remember stuff. It started as an internal blog at work. Many times I found my self wishing that I could access my work's Sharepoint blog from outside of work. Well that's not going to happen, so here I am.

The intention of this blog is to post bits of computer programming knowledge. I am a software engineer (developer/programmer, whatever) by day. I use C, C++, C#, and Python on a regular basis. Most of my blog entries will be about technologies related to these languages, although I may make an occasional disparaging remark about Perl.

I will be, at first, taking post from my internal work blog and re-publishing them here. After that I will mostly publish here, unless of course it is work related.