TypeError: _pfp__parse() got an unexpected keyword argument 'set_val'

[bannsec]

After creating the issue, checkboxes will appear where [] label exist in the
markdown. You can check/uncheck them to fill out the environment section.

Checklist

• I have included the [relevant portions of the] 010 template used that caused the bug
• I have filled out the environment section

Environment

Platform

• Windows
• Mac
• Linux
• Other (please specify)

Python Version

• Python 2.7
• Python 3.4
• Python 3.5
• Python 3.6
• Python 3.7
• Python 3.8
• Other (please specify)

Describe the bug

When attempting to load a base 010Editor template for ZIP format, I get the error:

TypeError: _pfp__parse() got an unexpected keyword argument 'set_val'

Stack Trace

2158930837---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _run(self, keep_successfull)
    996             # may change (e.g. compressed data)
--> 997             res = self._handle_node(self._ast, None, None, self._stream)
    998         except errors.InterpReturn as e:

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_file_ast(self, node, scope, ctxt, stream)
   1148                 continue
-> 1149             self._handle_node(child, scope, ctxt, stream)
   1150

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_while(self, node, scope, ctxt, stream)
   2372                 try:
-> 2373                     self._handle_node(node.stmt, scope, ctxt, stream)
   2374                 except errors.InterpBreak as e:

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_compound(self, node, scope, ctxt, stream)
   2186             for child in node.children():
-> 2187                 self._handle_node(child, scope, ctxt, stream)
   2188

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_if(self, node, scope, ctxt, stream)
   2317             # there should always be an iftrue
-> 2318             return self._handle_node(node.iftrue, scope, ctxt, stream)
   2319         else:

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_compound(self, node, scope, ctxt, stream)
   2186             for child in node.children():
-> 2187                 self._handle_node(child, scope, ctxt, stream)
   2188

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_decl(self, node, scope, ctxt, stream)
   1370                     field_res._pfp__interp = self
-> 1371                     field._pfp__init(stream)
   1372                     added_child = True

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _pfp__init(self, stream)
    149     def _pfp__init(self, stream):
--> 150         self._pfp__interp._handle_node(decls, ctxt=self, stream=stream)
    151

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_struct_decls(self, node, scope, ctxt, stream)
   1667                 # new context! (struct)
-> 1668                 self._handle_node(decl, scope, ctxt, stream)
   1669

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_if(self, node, scope, ctxt, stream)
   2314         self._dlog("handling if/ternary_op")
-> 2315         cond = self._handle_node(node.cond, scope, ctxt, stream)
   2316         if cond:

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_binary_op(self, node, scope, ctxt, stream)
   1833
-> 1834         left_val = self._handle_node(node.left, scope, ctxt, stream)
   1835         right_val = self._handle_node(node.right, scope, ctxt, stream)

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_binary_op(self, node, scope, ctxt, stream)
   1839
-> 1840         res = switch[node.op](left_val, right_val)
   1841

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in <lambda>(x, y)
   1819             "^": lambda x, y: x ^ y,
-> 1820             "&": lambda x, y: x & y,
   1821             "%": lambda x, y: x % y,

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/fields.py in __and__(self, other)
   1789         res = self.__class__()
-> 1790         res._pfp__set_value(self)
   1791         # takes care of promotion already

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/fields.py in _pfp__set_value(self, new_val)
   1612
-> 1613         promoted = self._pfp__promote(new_val)
   1614         self._pfp__value = promoted

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/fields.py in _pfp__promote(self, val)
   1585                 save_offset=False,
-> 1586                 set_val=False,
   1587             )

TypeError: _pfp__parse() got an unexpected keyword argument 'set_val'

During handling of the above exception, another exception occurred:

PfpError                                  Traceback (most recent call last)
<ipython-input-12-fba35c4b6192> in <module>
----> 1 dom = pfp.parse(data_file="./test.zip", template_file="ZIPTemplateAdv.bt")

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/__init__.py in parse(data, template, data_file, template_file, interp, debug, predefines, int3, keep_successful, printf)
     89         orig_filename=orig_filename,
     90         keep_successful=keep_successful,
---> 91         printf=printf,
     92     )
     93

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in parse(self, stream, template, predefines, orig_filename, keep_successful, printf)
    817         self._dlog("parsed template into ast")
    818
--> 819         res = self._run(keep_successful)
    820         res._pfp__finalize()
    821         return res

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _run(self, keep_successfull)
   1022                         else more_info
   1023                     ),
-> 1024                     traceback,
   1025                 )
   1026

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
    693                 value = tp()
    694             if value.__traceback__ is not tb:
--> 695                 raise value.with_traceback(tb)
    696             raise value
    697         finally:

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _run(self, keep_successfull)
    995             # it is important to pass the stream in as the stream
    996             # may change (e.g. compressed data)
--> 997             res = self._handle_node(self._ast, None, None, self._stream)
    998         except errors.InterpReturn as e:
    999             # TODO handle exit/return codes (e.g. return -1)

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_file_ast(self, node, scope, ctxt, stream)
   1147                     is_forward_declared_struct(child):
   1148                 continue
-> 1149             self._handle_node(child, scope, ctxt, stream)
   1150
   1151         ctxt._pfp__process_fields_metadata()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_while(self, node, scope, ctxt, stream)
   2371             if node.stmt is not None:
   2372                 try:
-> 2373                     self._handle_node(node.stmt, scope, ctxt, stream)
   2374                 except errors.InterpBreak as e:
   2375                     break

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_compound(self, node, scope, ctxt, stream)
   2185         try:
   2186             for child in node.children():
-> 2187                 self._handle_node(child, scope, ctxt, stream)
   2188
   2189         # in case a return occurs, be sure to pop the scope

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_if(self, node, scope, ctxt, stream)
   2316         if cond:
   2317             # there should always be an iftrue
-> 2318             return self._handle_node(node.iftrue, scope, ctxt, stream)
   2319         else:
   2320             if node.iffalse is not None:

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_compound(self, node, scope, ctxt, stream)
   2185         try:
   2186             for child in node.children():
-> 2187                 self._handle_node(child, scope, ctxt, stream)
   2188
   2189         # in case a return occurs, be sure to pop the scope

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_decl(self, node, scope, ctxt, stream)
   1369
   1370                     field_res._pfp__interp = self
-> 1371                     field._pfp__init(stream)
   1372                     added_child = True
   1373                 else:

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _pfp__init(self, stream)
    148
    149     def _pfp__init(self, stream):
--> 150         self._pfp__interp._handle_node(decls, ctxt=self, stream=stream)
    151
    152     cls_members = {

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_struct_decls(self, node, scope, ctxt, stream)
   1666             for decl in node.decls:
   1667                 # new context! (struct)
-> 1668                 self._handle_node(decl, scope, ctxt, stream)
   1669
   1670             ctxt._pfp__process_fields_metadata()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_if(self, node, scope, ctxt, stream)
   2313         """
   2314         self._dlog("handling if/ternary_op")
-> 2315         cond = self._handle_node(node.cond, scope, ctxt, stream)
   2316         if cond:
   2317             # there should always be an iftrue

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_binary_op(self, node, scope, ctxt, stream)
   1832         }
   1833
-> 1834         left_val = self._handle_node(node.left, scope, ctxt, stream)
   1835         right_val = self._handle_node(node.right, scope, ctxt, stream)
   1836

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_node(self, node, scope, ctxt, stream)
   1104             )
   1105
-> 1106         res = self._node_switch[node.__class__](node, scope, ctxt, stream)
   1107
   1108         self._log.dec()

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in _handle_binary_op(self, node, scope, ctxt, stream)
   1838             raise errors.UnsupportedBinaryOperator(node.coord, node.op)
   1839
-> 1840         res = switch[node.op](left_val, right_val)
   1841
   1842         if type(res) is bool:

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/interp.py in <lambda>(x, y)
   1818             "|": lambda x, y: x | y,
   1819             "^": lambda x, y: x ^ y,
-> 1820             "&": lambda x, y: x & y,
   1821             "%": lambda x, y: x % y,
   1822             ">": lambda x, y: x > y,

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/fields.py in __and__(self, other)
   1788     def __and__(self, other):
   1789         res = self.__class__()
-> 1790         res._pfp__set_value(self)
   1791         # takes care of promotion already
   1792         res &= other

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/fields.py in _pfp__set_value(self, new_val)
   1611             raise errors.UnmodifiableConst()
   1612
-> 1613         promoted = self._pfp__promote(new_val)
   1614         self._pfp__value = promoted
   1615

~/.virtualenvs/hashcrack/lib/python3.6/site-packages/pfp/fields.py in _pfp__promote(self, val)
   1584                 six.BytesIO(raw),
   1585                 save_offset=False,
-> 1586                 set_val=False,
   1587             )
   1588         else:

PfpError: TypeError: _pfp__parse() got an unexpected keyword argument 'set_val'
Exception at ZIPTemplateAdv.bt:403

To Reproduce

• pip install pfp
• ipython
• import pfp
• dom = pfp.parse(data_file="./test.zip", template_file="ZIPTemplateAdv.bt")

ZIPTemplateAdv.bt.zip

The relevant line in the template appears to be:

 if ((frFlags & FLAG_Encrypted) && ( frFlags & FLAG_StrongEncrypted))

Expected Behavior

Get dom object.

Implementation/Fix Notes/Thoughts

No idea. Guessing it has something to do with the mutl-level boolean logic.

[d0c-s4vage]

Thanks for reporting this issue!

[d0c-s4vage]

the set_val problem may already be fixed - I'll add more details once I triage/reproduce it!

[bannsec]

Oh. Just tried it with master (instead of pypi release) and it does not error. It does seem to print out some number upon parsing (not sure what that is). However, beyond that it seems like it's working in current master.

This library is super helpful dude. Keep up the good work :-)

[d0c-s4vage]

Thank you! I really like this project. It's not done yet, and there's tons of improvements that can be made, but it's really fun to work on. Also, I looked up when the set_val bug was fixed - #113 fixed it.

…

On Sun, Jan 19, 2020, 09:16 bannsec ***@***.***> wrote: Oh. Just tried it with master (instead of pypi release) and it does not error. It does seem to print out some number upon parsing (not sure what that is). However, beyond that it seems like it's working in current master. This library is super helpful dude. Keep up the good work :-) — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#118?email_source=notifications&email_token=ABG2WYQBBG3NJ6I2H5OWJ33Q6SDHRA5CNFSM4KIXGRL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJKXKSA#issuecomment-576025928>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABG2WYVHTHTTOHXEMB35C7LQ6SDHRANCNFSM4KIXGRLQ> .

[d0c-s4vage]

@bannsec:

It does seem to print out some number upon parsing

Line 226 in the ZIPTemplateAdv.bt prints the header id:

	Printf("%d", efHeaderID);

Remove that line or comment it out to keep it from printing the number.