Questions tagged [pickle]
Pickle is an object serialization module for Python. Use this tag together with the Python tag for questions related to storing or loading objects with Pickle.
pickle
4,991
questions
0
votes
0
answers
14
views
Loading Joblib model in Airflow dag
I'm trying to use joblib inside an airflow dag, when loading a model file I run joblib.load(model_name) but I get an error
File "/usr/local/lib/python3.9/pickle.py", line 331, in ...
- 7,319
0
votes
0
answers
13
views
Store/load Scikit Learn objects (Pipeline, ColumnTransformer) without different version considerations?
I need to be able to somehow store and reload a Scikit Learn pipeline in a way that breaking changes between different Scikit Learn versions aren't as much of a concern as they are when I use e.g. ...
- 2,081
0
votes
0
answers
12
views
dill: how to skip ctypes pointers
I need to dump and load my object which includes objects from third party modules over which I have no control. These objects contain ctypes pointers that dill (pickle) cannot save:
ValueError: ctypes ...
- 775
-1
votes
1
answer
37
views
Loading python classes using pickle gives error: init missing arguments
I have a class which contains some basic information as well as other classes, when saving this class
the attribute buildings gives error upon loading
TypeError: Building.__init__() missing 2 required ...
- 1
0
votes
1
answer
44
views
Python "r+" mode reads nothing after writing with Pickle library
I have a file that I want to read a list from and then write an updated list to it by using the pickle library. But after that it reads nothing from the file. I have tried using other people's ...
- 3
0
votes
0
answers
33
views
Unable to unpickle file into a dataframe
import pandas as pd
import pickle as pic
file_obj = open('D:/BAML/Production/EQFICC-Python/FCCV/2005-10-09_RFCV.pickle', 'rb')
data = pic.load(file_obj)
Error :
runfile('D:/BAML/Production/...
- 1
0
votes
0
answers
22
views
Convert trained and pickled sklearn ColumnTransformer from sklearn 1.1.2 to 1.5.1
I have a Scikit Learn ColumnTransformer containing a number of other transformers (passthrough, a custom transformer and a OneHotEncoder) that has been "trained" (transformer.fit() has been ...
- 2,081
0
votes
0
answers
34
views
Recover a corrupted pickle file
I have a corrupted pickle file. I follow this to try to recover my file: how to read corrupted pickle file
I read and try to understand how the file work to complete the file and as a result, I ...
- 31
0
votes
0
answers
15
views
Pickle load to CPU of an internal model
lets state clearly my problem. I have a feature_extractor.pkl file that denotes a FeatureExtractor object which object has inside a BERT model like the following class
class BERTFeatureExtractor:
...
- 1
0
votes
0
answers
75
views
unable to unpickle a file on windows that was pickle on MAC
I'm trying to unpickle a file called nlp.pk, it was pickled on a MAC and now I'm trying to unpickle it on windows.
below is the piece of code I'm using to unpickle the file
_dependency_file_path = {
...
- 54
-2
votes
1
answer
86
views
Efficiently storing large numbers of large integers from Python to file
I have a large number of large integers, e.g. generated using:
import random
low = 10 ** 10
high = 10 ** 100
numbers = sorted([random.randint(low, high) for _ in range(10000000)])
It can be assumed ...
- 481
0
votes
0
answers
30
views
In Seq2Seq Models with encoders and decoders; How do I fix, "Data Cardinality is Ambiguous" when it refers to the target data?
I have an error with my chatbot that debugging and chatbots can't help. I can't find documentation on this at all or maybe I'm looking in the wrong area. I built a seq2seq chatbot model using ...
0
votes
0
answers
51
views
Critical Vulnerability detection in pandas Library by Fortify due to read_pickle function
Our company utilizes the pandas library extensively in our software. However, Fortify, our security analysis tool,** flags pandas as having a critical vulnerability due to the read_pickle function**, ...
7
votes
0
answers
227
views
Memory use of pickling/unpickling enum in Python
While investigating the memory use of an application that exchanges message through PyZMQ using tracemalloc, I noticed a weird behavior of pickling and unpickling enums: memory looks like it is ...
0
votes
0
answers
41
views
How can I store a Vector store var some how and query from it?
Note: Newbie to LLM's
I want to store a vector store show, like in a pickle file or something, and use it for ConversationalRetrievalChain ..
# storing embeddings in the vector store
vectorstore = ...
