haystack 0.8

Search C Structures in a process' memory

HOWTO:
------


Run as command line :

$ sudo haystack --pid 26725 sslsnoop.ctypes_openssh.session_state refresh 0xb8b70d18 > instance.pickled
$ sudo haystack --pid 26725 <your ctypes Structure > search > instance.pickled

Or as a Graphical GUI ( Qt4 )

$ haystack-gui

Run in a python script :
>>> import haystack
>>> haystack.findStruct( pid , 'ctypes.c_int')
>>> haystack.findStruct( pid , 'ctypes_example.big_struct')

It's easy to add new structures (check ctypeslib or do it by hand or check sslsnoop on github)

a) Your class must extend haystack.model.LoadableMembers.
b) You must give your class a completed _fields_ (with one _ ), like all ctypes.Structure
c) You can add an expectedValues dict() to your class to refine your search
d) call model.registerModule(sys.modules[__name__])

Advanced use : You can override methods to fine tune some validation or loading

The global algoritm :
a) A ctypes structure is mapped at a memory address.
b) The method loadMembers id called.
c) The method isValid is called on self.
d) A validation test is done for each members, it's expected values and memory space validity (pointers) are tested.
The validation does not recurse.
e) Each members is then 'loaded' to local space.
If the value is a pointer or a model.LoadableMembers type, it's recursively Loaded. ( and validated).
If the recursive loading fails, the calls fails. bye-bye.
f) see next offset, goto a)

GUI usage:
----------

You need to have a process memory dump :

$ sudo haystack-dump dump 26725 dumps/myssh.dump

Then you can open it in the GUI :

$ haystack-gui # ( and Ctrl-O , click click)
$ hasytack-gui --dumpfile dumps/myssh.dump

The dump file format is a simple tar containing each memory map :
* content in a file named after it's start/end addresses ( 0x000700000-0x000800000 )
* python haystack.model.MemoryMapping object pickled ( 0x000700000-0x000800000.pickled )

You can the search a structure into that memory mapping ( [heap] for now ).
You have to import your extensions before that ( try sslsnoop.ctypes_openssh ) to have them listed in the search dialog.


Extension examples :
---------------------
@ see sslsnoop in the Pypi repo. openssl and nss structures are generated.
@ see ctypes-kernel on my github. Linux kernel structure are generated. (VMM is abitch)


Pseudo Example :

from haystack.model import LoadableMembers, RangeValue, NotNull

class OpenSSLStruct(LoadableMembers):
pass

class RSA(OpenSSLStruct):
''' rsa/rsa.h '''
_fields_ = [
("pad", ctypes.c_int),
("version", ctypes.c_long),
("meth",ctypes.POINTER(BIGNUM)),#const RSA_METHOD *meth;
("engine",ctypes.POINTER(ENGINE)),#ENGINE *engine;
('n', ctypes.POINTER(BIGNUM) ), ## still in ssh memap
('e', ctypes.POINTER(BIGNUM) ), ## still in ssh memap
('d', ctypes.POINTER(BIGNUM) ), ## still in ssh memap
('p', ctypes.POINTER(BIGNUM) ), ## still in ssh memap
('q', ctypes.POINTER(BIGNUM) ), ## still in ssh memap
('dmp1', ctypes.POINTER(BIGNUM) ),
('dmq1', ctypes.POINTER(BIGNUM) ),
('iqmp', ctypes.POINTER(BIGNUM) ),
("ex_data", CRYPTO_EX_DATA ),
("references", ctypes.c_int),
("flags", ctypes.c_int),
("_method_mod_n", ctypes.POINTER(BN_MONT_CTX) ),
("_method_mod_p", ctypes.POINTER(BN_MONT_CTX) ),
("_method_mod_q", ctypes.POINTER(BN_MONT_CTX) ),
("bignum_data",ctypes.POINTER(ctypes.c_ubyte)), ## moue c_char_p ou POINTER(c_char) ?
("blinding",ctypes.POINTER(BIGNUM)),#BN_BLINDING *blinding;
("mt_blinding",ctypes.POINTER(BIGNUM))#BN_BLINDING *mt_blinding;
]
expectedValues={
"pad": [0],
"version": [0],
"references": RangeValue(0,0xfff),
"n": [NotNull],
"e": [NotNull],
"d": [NotNull],
"p": [NotNull],
"q": [NotNull],
"dmp1": [NotNull],
"dmq1": [NotNull],
"iqmp": [NotNull]
}
def loadMembers(self, mappings, maxDepth):
print 'example'
if not LoadableMembers.loadMembers(self, mappings, maxDepth):
log.debug('RSA not loaded')
return False
return True

# register to haystack
model.registerModule(sys.modules[__name__])


not so FAQ :
============

What does it do ?:
------------------
The basic functionnality is to search in a process' memory maps for a specific C Structures.

How do it knows that the structures is valid ? :
------------------------------------------------
You add some constraints ( expectedValues ) on the fields. Pointers are also a good start.

Where does the idea comes from ? :
-----------------------------------
use http://www.hsc.fr/ressources/breves/passe-partout.html.fr to get keys
use http://pauldotcom.com/2010/10/tsharkwireshark-ssl-decryption.html
or http://www.rtfm.com/ssldump/ to read streams
use scapy, because it's fun ? but we need IP reassembly .
pynids could be more useful...
dsniff is now in python ?
flowgrep
use python.


What are the dependencies ? :
----------------------------
python-ptrace