logo
down
shadow

Applying literal_eval on string of lists of POS tags gives ValueError


Applying literal_eval on string of lists of POS tags gives ValueError

By : user2173178
Date : October 21 2020, 08:10 PM
wish helps you In a pandas column I have list of POS tags as string. I thought this must be string because print(dataset['text_posTagged'][0][0]) prints [. , Parse only non-null rows. You can drop the lambda.
code :
m = dataset['text_posTagged'].notna()
dataset.loc[m, 'text_posTagged'] = (
    dataset.loc[m, 'text_posTagged'].apply(literal_eval))
dataset.loc[m, 'text_posTagged'] = pd.eval(dataset.loc[m, 'text_posTagged'])


Share : facebook icon twitter icon
ValueError: malformed string when using ast.literal_eval

ValueError: malformed string when using ast.literal_eval


By : D. Quesada
Date : March 29 2020, 07:55 AM
should help you out The reason this doesn’t work on Python 2 lies in its implementation of literal_eval. The original implementation only performed number evaluation for additions and subtractions when the righth operand was a complex number. This is syntactically necessary for complex numbers to be expressed as a literal.
This was changed in Python 3 so that it supports any kind of valid number expression to be on either side of the addition and subtraction. However, the use of literal_eval is still restricted to additions and subtractions.
code :
import ast, operator

binOps = {
    ast.Add: operator.add,
    ast.Sub: operator.sub,
    ast.Mult: operator.mul,
    ast.Div: operator.div,
    ast.Mod: operator.mod
}

def arithmeticEval (s):
    node = ast.parse(s, mode='eval')

    def _eval(node):
        if isinstance(node, ast.Expression):
            return _eval(node.body)
        elif isinstance(node, ast.Str):
            return node.s
        elif isinstance(node, ast.Num):
            return node.n
        elif isinstance(node, ast.BinOp):
            return binOps[type(node.op)](_eval(node.left), _eval(node.right))
        else:
            raise Exception('Unsupported type {}'.format(node))

    return _eval(node.body)
>>> arithmeticEval('4+2')
6
>>> arithmeticEval('4*1+2*6/3')
8
ValueError: malformed string using ast.literal_eval

ValueError: malformed string using ast.literal_eval


By : silvio
Date : March 29 2020, 07:55 AM
fixed the issue. Will look into that further You should not use ast.literal_eval() on JSON data. JSON and Python literals may look like the same thing, but they are very much not.
In this case, your data contains a boolean flag, set to false in JSON. A proper Python boolean uses title-case, so False:
code :
>>> import json, ast
>>> s = '{"no_sell_or_sort": false, "size": 20}'
>>> json.loads(s)
{u'no_sell_or_sort': False, u'size': 20}
>>> ast.literal_eval(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 80, in literal_eval
    return _convert(node_or_string)
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 63, in _convert
    in zip(node.keys, node.values))
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 62, in <genexpr>
    return dict((_convert(k), _convert(v)) for k, v
  File "/Users/mj/Development/Library/buildout.python/parts/opt/lib/python2.7/ast.py", line 79, in _convert
    raise ValueError('malformed string')
ValueError: malformed string
response_item = ast.literal_eval(json.dumps(response_item, ensure_ascii=False).encode('utf8'))
python: ast.literal_eval() gives ValueError: malformed string

python: ast.literal_eval() gives ValueError: malformed string


By : George Navas
Date : March 29 2020, 07:55 AM
this one helps. The string '(ENG,80),(PHY,65),(CHEM,80)' is not a valid Python literal. Presumably you want ENG, PHY and CHEM to be treated as strings, but they are not quoted to be valid Python string literals.
Just parse this part manually:
code :
N = {' Marie ': ' (ENG,80) (PHY,65) (CHEM,80) '} 
value = N.values()[0]
output = [(token, int(num)) for tup in value.split() for token, num in (tup.strip('()').split(','),)]
>>> N = {' Marie ': ' (ENG,80) (PHY,65) (CHEM,80) '}
>>> value = N.values()[0]
>>> [(token, int(num)) for tup in value.split() for token, num in (tup.strip('()').split(','),)]
[('ENG', 80), ('PHY', 65), ('CHEM', 80)]
Malformed String ValueError ast.literal_eval() with String representation of Tuple

Malformed String ValueError ast.literal_eval() with String representation of Tuple


By : Gomathi Ganesan
Date : March 29 2020, 07:55 AM
it helps some times ast.literal_eval (located in ast.py) parses the tree with ast.parse first, then it evaluates the code with quite an ugly recursive function, interpreting the parse tree elements and replacing them with their literal equivalents. Unfortunately the code is not at all expandable, so to add Decimal to the code you need to copy all the code and start over.
For a slightly easier approach, you can use ast.parse module to parse the expression, and then the ast.NodeVisitor or ast.NodeTransformer to ensure that there is no unwanted syntax or unwanted variable accesses. Then compile with compile and eval to get the result.
ast.literal_eval ValueError('malformed string')

ast.literal_eval ValueError('malformed string')


By : MR-Ingenuity
Date : March 29 2020, 07:55 AM
To fix this issue I have a string of the form {"Top":[{"A":1,"B":721.0,"C":false}]} which I would like to convert into a Python collection. , false is not accepted by ast.literal_eval.
Related Posts Related Posts :
  • Submitting login form with scrapy
  • How do i edit the favicon in the Browsable API in Django REST framework?
  • multiprocessing.Pool.map_async doesn't seem to... do anything at all?
  • Python Selenium: Stale Element Reference Exception Error
  • Datetime conversion - How to extract the inferred format?
  • Import YAML variables automatically?
  • How to create a powershell shortcut for my python file
  • Python's 'set' operator doesn't work with numpy.nan
  • Pass object fields and one2many fields on same method - Odoo v8
  • Select columns based on column name and location in Pandas
  • Standardizing timeseries in Pandas using interpolation
  • How many tweets can be collected?
  • how format specifier taking value while tuple list is passed
  • How to print a numpy array with data type?
  • Timeout child thread for python3
  • How can I regroup a dataframe and accumulate a colume's values?
  • Bulk Insert into SQL Server with Python not working
  • Removing last rows of each group based on condition in a pandas dataframe
  • Why the css file can not be found in Django template?
  • targeting center of mass - scipy / numpy
  • Foursquare - get tips from VENUE_ID
  • Unpack a dictionary to format
  • encoding special characters in python2
  • Replacing integers with NaN results in the entire column becoming float dtype
  • Python 3.6 - BeautifulSoup4, parse table AttributeError: ResultSet object has no attribute 'findAll'
  • Convert panda date list to python list of date strings
  • escape response from Scrapy to parse json
  • How to create a same dropdown menu for different labels?
  • Why are some python variables uppercase whereas others are lowercase?
  • Machine Learning, What are the common techniques for feature engineering and presenting the model?
  • Modify value of a Django form field during clean() and validate again
  • Heroku Django app can't start up -- 'No module named site'
  • Getting list of dates (excluding weekends)
  • Im trying to create the regular expression to include the text and not the href
  • Python file.readline(2) reads first 2 charectars
  • Groupby with handling empty bin in python
  • Modifying Gcode
  • calling a value in a dictionary within a dictionary (reading a json file)
  • Bouncing ball invalid syntax why is that?
  • Python making a counter
  • Python rstrip and split
  • What does the String mean in numpy.r_?
  • How to correctly extend variable __all__ in a __init__.py?
  • Python behaves weird with piped input
  • Python 3 two dimensional list comprehension
  • How to slice image by broadcasting slices? Error: 'only integer scalar arrays can be converted to a scalar index' in pyt
  • (Python Beginner) Need a start on classes
  • IndexError: At least one sheet must be visible
  • How to solve a system of linear equations over the nonnegative integers?
  • Pandas keep the most complete rows
  • "List index out of range" error in Python Memory Match game
  • Numpy: how to use argmax results to get the actual max?
  • Google Cloud Dataflow can't import 'google.cloud.datastore'
  • Calculate pandas DataFrame column by custom routine which accepts dictionary as input
  • Connect to a Class Method by it's method name holded into a var in a for loop in python
  • PyQt5 signals and threading.Timer
  • Replace 2 characters in a string in python
  • Passing command line arguments from a folder script to a file script
  • Understand the syntaxe X[Y == c] in Numpy
  • Optimize beginner python script about substring replacement
  • shadow
    Privacy Policy - Terms - Contact Us © voile276.org