Bokeh: ValueError: Out of range float values are not JSON compliant

Question

I came across this discussion (from a year ago): https://github.com/bokeh/bokeh/issues/2392

I also saw the white screen without any errors..and then i tried to take a small subset of 2 columns and tried the below:

Since pandas just gets a bunch of rows with empty data in there as well, I tried dropna.. this resulted in there being no data at all. So instead I just specified the rows that should go into the df (hence the df = df.head(n=19) line)

import pandas as pd
from bokeh.plotting import figure, output_file, show

df = pd.read_excel(path,sheetname,parse_cols="A:B")
df = df.head(n=19)
print(df)
rtngs = ['iAAA','iAA+','iAA','iAA-','iA+','iA','iA-','iBBB+','iBBB','iBBB-','iBB+','iBB','iBB-','iB+','iB','iB-','NR','iCCC+']
x= df['Score']
output_file("line.html")

p = figure(plot_width=400, plot_height=400, x_range=(0,100),y_range=rtngs)

# add a circle renderer with a size, color, and alpha
p.circle(df['Score'], df['Rating'], size=20, color="navy", alpha=0.5)

# show the results
#output_notebook()
show(p)

df:

   Rating  Score
0    iAAA   64.0
1    iAA+   33.0
2     iAA    7.0
3    iAA-   28.0
4     iA+   36.0
5      iA   62.0
6     iA-   99.0
7   iBBB+   10.0
8    iBBB   93.0
9   iBBB-   91.0
10   iBB+   79.0
11    iBB   19.0
12   iBB-   95.0
13    iB+   26.0
14     iB    9.0
15    iB-   26.0
16     NR   49.0
17  iCCC+   51.0
18   iAAA   18.0

The above is showing me an output within the notebook, but still throws : ValueError: Out of range float values are not JSON compliant

And also it doesn't (hence?) produce the output file as well. How do I get rid of this error for this small subset? Is it related to NaN values? Would that also solve the 'white screen of death' issue for the larger dataset?

Thanks vm for taking a look!

In case you would like to see the entire error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-12-4fa6b88aa415> in <module>()
     16 # show the results
     17 #output_notebook()
---> 18 show(p)

C:\Users\x\AppData\Local\Continuum\Anaconda3\lib\site-packages\bokeh\io.py in show(obj, browser, new)
    300     if obj not in _state.document.roots:
    301         _state.document.add_root(obj)
--> 302     return _show_with_state(obj, _state, browser, new)
    303 
    304 

C:\Users\x\AppData\Local\Continuum\Anaconda3\lib\site-packages\bokeh\io.py in _show_with_state(obj, state, browser, new)
    310 
    311     if state.notebook:
--> 312         comms_handle = _show_notebook_with_state(obj, state)
    313         shown = True
    314 

C:\Users\x\AppData\Local\Continuum\Anaconda3\lib\site-packages\bokeh\io.py in _show_notebook_with_state(obj, state)
    334         comms_target = make_id()
    335         publish_display_data({'text/html': notebook_div(obj, comms_target)})
--> 336         handle = _CommsHandle(get_comms(comms_target), state.document, state.document.to_json())
    337         state.last_comms_handle = handle
    338         return handle

C:\Users\x\AppData\Local\Continuum\Anaconda3\lib\site-packages\bokeh\document.py in to_json(self)
    792         # this is a total hack to go via a string, needed because
    793         # our BokehJSONEncoder goes straight to a string.
--> 794         doc_json = self.to_json_string()
    795         return loads(doc_json)
    796 

C:\Users\x\AppData\Local\Continuum\Anaconda3\lib\site-packages\bokeh\document.py in to_json_string(self, indent)
    785         }
    786 
--> 787         return serialize_json(json, indent=indent)
    788 
    789     def to_json(self):

C:\Users\x\AppData\Local\Continuum\Anaconda3\lib\site-packages\bokeh\core\json_encoder.py in serialize_json(obj, encoder, indent, **kwargs)
     97         indent = 2
     98 
---> 99     return json.dumps(obj, cls=encoder, allow_nan=False, indent=indent, separators=separators, sort_keys=True, **kwargs)

C:\Users\x\AppData\Local\Continuum\Anaconda3\lib\json\__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    235         check_circular=check_circular, allow_nan=allow_nan, indent=indent,
    236         separators=separators, default=default, sort_keys=sort_keys,
--> 237         **kw).encode(obj)
    238 
    239 

C:\Users\x\AppData\Local\Continuum\Anaconda3\lib\json\encoder.py in encode(self, o)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

C:\Users\x\AppData\Local\Continuum\Anaconda3\lib\json\encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258 
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

ValueError: Out of range float values are not JSON compliant

I encountered this same error message while using the requests.post(url, json=data) in which data contained some NaNs. The solution is to clean up the data and fix all the NaN such as putting quotes around it, or changing to null or zeros, etc. — John Henckel, Commented May 11, 2023 at 14:57
Oh! and one more thing. If you want to test if some float is JSON compatible, do not rely on math.isnan() because it will return False on infinities. APPARENTLY the developers of Python think that Infinity is a number 😲 Instead you should write if not math.isfinite(x): x = None in order to be JSON compatible. — John Henckel, Commented May 11, 2023 at 17:00

ragesz · Accepted Answer · 2016-12-18 21:31:14Z

I had the same error and I debugged the problem: I had NaN values in my plotted dataset and bokeh's serialize_json() function (in /core/json_encoder.py) does not allow NaN values (I don't know why...). In the return part of this function there is the allow_nan=False argument in json.dumps() :(( The problem occurs only at the io part of bokeh process when the output file is generated (it calls the above serialize_json() function).

So you have to replace NaN values in your dataframe, eg.:

df = df.fillna('')

Nice day! :)

bigreddot · Accepted Answer · 2016-12-26 23:27:06Z

4

NaN support will be better supported when this Pull Request to add a binary array serialization option is merged. This should be available in Bokeh 0.12.4 in January 2017. Bokeh does not use allow_nan in the python JSON encoder, because that is not standard — nan and inf are not part of the official JSON specification (an egregious oversight IMO, but out of our control)

edited Dec 26, 2016 at 23:27

answered Dec 19, 2016 at 15:54

bigreddot

34.4k5 gold badges72 silver badges126 bronze badges

Add a comment |

Ignatios Chatzidiamantis · Accepted Answer · 2018-03-03 16:59:53Z

Well it isn't exactly an answer to your question it's more like my experience working with bokeh for a week. In my case trying to make a plot like the Texas example from bokeh..... After a lot of frustration i noticed that bokeh or json or whatever when encounters the first value of the list (myList) to be plotted to be a NaN it refuses to plot giving the message

ValueError: Out of range float values are not JSON compliant

if i change the first value of the list (myList[0]) to float it works fine even if it contains NaN's to other positions. Taking this in account someone who understands how these things work will propose an answer. Mine is to restruct your data so that the first value isn't a nan.

id101112 · Accepted Answer · 2017-11-02 22:09:24Z

2

After removing the NAN values, there might be infinite value, Trace the whole dataset it might have some infinite values as inf remove those infinite values some how, then it should work.

df['column'].describe()

then if you find any inf value remove those rows with

df = df[~df.isin([np.nan, np.inf, -np.inf]).any(1)]

reference: solution here

edited Nov 2, 2017 at 22:09

answered Nov 2, 2017 at 22:01

id101112

1,0423 gold badges16 silver badges29 bronze badges

Add a comment |

Michele · Accepted Answer · 2019-03-19 18:54:11Z

1

I bumped into this problem and I realized it was happening because one column of my Dataframe was filled only with NaNs.

You could instead put it to another value, e.g.:

df['column'] = np.zeros(len(df))

answered Mar 19, 2019 at 18:54

Michele

2,9352 gold badges22 silver badges31 bronze badges

Add a comment |

Marvin M. Agüero-Torales · Accepted Answer · 2020-06-18 12:05:48Z

I had this error in this line:

save(plot_lda, 'tsne_lda_viz_{}_{}_{}_{}_{}_{}_{}.html'.format(
    num_qualified_tweet, n_topics, threshold, n_iter, num_example, n_top_words, end_date))

I worked use this repo as baseline: https://github.com/ShuaiW/twitter-analysis/blob/master/topic_tweets.py (mine)

And, i solved this with this code (hope this will useful for others):

  for i in range(X_topics.shape[1]):
    topic_coord[i, 0] = 0 if np.isnan(topic_coord[i, 0]) else topic_coord[i, 0]
    topic_coord[i, 1] = 0 if np.isnan(topic_coord[i, 1]) else topic_coord[i, 1]
    plot_lda.text(topic_coord[i, 0], topic_coord[i, 1], [topic_summaries[i]])

The key is:

var = 0 if np.isnan(number) else number

hassanzadeh.sd · Accepted Answer · 2020-09-20 09:36:47Z

1

I have this issue and solved with clean my dataset

check your data set and change null records value.

answered Sep 20, 2020 at 9:36

hassanzadeh.sd

3,3631 gold badge20 silver badges28 bronze badges

Add a comment |

Ky-Di · Accepted Answer · 2022-05-27 18:18:36Z

0

If anyone else comes across this answer you can specify a parameter on the read_excel method to not use default NA values

pd.read_excel(path, sheetname, parse_cols="A:B", keep_default_na=False)

Reference: https://pandas.pydata.org/docs/reference/api/pandas.read_excel.html

If keep_default_na is False, and na_values are not specified, no strings will be parsed as NaN.

answered May 27, 2022 at 18:18

Ky-Di

1

Add a comment |

Peter Barmettler · Accepted Answer · 2023-08-02 16:08:06Z

If you can't or don't want to remove nan values from your dataset you can patch bokeh to use a custom encoder


from bokeh.core import json_encoder

class PayloadEncoder(json_encoder.PayloadEncoder):
    def __init__(self, *, buffers, threshold: int = 100,
            indent: int | None = None, separators: tuple[str, str] | None = None):
        super().__init__(buffers=buffers,threshold=threshold,indent=indent,separators=separators)
        self.allow_nan = True

json_encoder.PayloadEncoder = PayloadEncoder

This might have unintended side effects. Only recommended for experimentation in jupyter notebooks.

Collectives™ on Stack Overflow

Bokeh: ValueError: Out of range float values are not JSON compliant

9 Answers 9

Not the answer you're looking for? Browse other questions tagged
python
bokeh
or ask your own question.

Linked

Hot Network Questions

Collectives™ on Stack Overflow

9 Answers 9

Not the answer you're looking for? Browse other questions tagged pythonbokeh or ask your own question.

Linked

Related

Not the answer you're looking for? Browse other questions tagged
python
bokeh
or ask your own question.