-
Notifications
You must be signed in to change notification settings - Fork 19.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Removing layers with layers.pop() doesn't work? #2371
Comments
If you want to fine-tune this model, you should add new dense layer after popping the original one. |
The original author does that; here is the notebook: https://github.com/iamaaditya/VQA_Demo/blob/master/Visual_Question_Answering_Demo_in_python_notebook.ipynb Image features are supposed to be merged with language features, but it doesn't come to that because of the error I mentioned. The issue is, the model still uses the popped layers, which can be verified by plotting it. If I comment out "model = model.model" (line 21 of keras/utils/visualize_util.py), the right model without the last layers is plotted. But I don't know what it means or how to let Keras know the layers have changed. Additionally, I just verified it doesn't happen with Keras 0.3.1, the layers were dropped correctly there. |
I actually ran into this recently, and I believe that this is an issue with how A method to pop layers off and manage all the links correctly is needed, but until then I believe this would work: (my model compiles, but I haven't tried training it yet)
|
Thank you. I downgraded to 0.3.1 where it works, so can't try your fix right now. I'm not sure it's all there is, because if I'm not mistaken As a side note, |
@fchollet It could be possible to add into a Sequential model the possibility to pop the last layer and updating the output and outbound_nodes? |
@albertomontesg like this?
|
@joelthchao Yes. That was exactly what I was thinking. |
@joelthchao seems to work fine, indeed. Thanks! P.S. This particular example runs into #2386 after that, so it works under 0.3.1 only. |
@joelthchao Thanks for the idea. This issue can be considered close. |
can confirm I'm still having this issue on keras 1.0.6. |
I'm having a related problem #3310. |
Same. |
I encountered the same problem and I managed to pop the layers like this: from convnetskeras.convnets import AlexNet Hope this helps! |
@joelthchao I wanna use this pretrained vgg19.
i have only 8 labels to classlify.
Do you give me some advices? Thanks |
@alyato No need to pop, just find the layer you want to connect to, connect to another layer and make another new model with correct input and output. |
Snippet for your question: from keras.models import Model
from keras.layers import Dense, Input
# Assume we have a pretrained model
net_input = Input(shape=(10))
net = Dense(4) (net_input)
net_output = Dense(1) (net)
model1 = Model(net_input, net_output)
# Get input
new_input = model1.input
# Find the layer to connect
hidden_layer = model1.layers[-2].output
# Connect a new layer on it
new_output = Dense(2) (hidden_layer)
# Build a new model
model2 = Model(new_input, new_output) And this is the result
|
Also, there's an available method |
@joelthchao Thanks. I run it well. But I also don't know how to get the |
@alyato Sort of, I use |
Hi, I am trying to append vggnet-16 to an existing ResNet model, but after executing Pop() written by you, the output shape value is |
@srv902 Is it a sequential model or functional API model? |
@joelthchao It is a functional API model. |
@srv902 If the layer you want to connect to has multiple outputs, then you can only choose one output to connect. (I bet you are trying to connect to residual block.) BTW, you can provide a code snippet to help others to understand your problem. |
Still has this problem! |
Below method works:
reference here |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed. |
I can confirm that keras 1.1.0 model.pop() does not remove the last layer. Found this thread due to running into the issue myself. Code with the error:
Edit:I got the pop to work by calling |
I think I have a similar problem. I am running Keras 2.0.6 with Python 3.6.2 and Tensorflow-gpu 1.3.0. I run this code after having hand built a vgg16 architecture and loaded the weights:
And when I check the graph in Tensorboard I see (check top left of attached picture) dense_3 connected to dropout_2 but dangling by itself. And then next to it I see dense_4, also connected to dropout_2. Curiously, I get a low accuracy when running this on the Kaggle cats vs dog competition where I hover around 90% whilst others running this code (it's adapted from fast.ai) on top of Theanos easily get 97%. Perhaps my accuracy problem comes from somewhere else, but I still don't think dense_3 should be dangling there and I am wondering if this could be the source of my precision problem. Secondary question: how could I definitely disconnect and remove dense_3? |
Have you tried this: model = self.model #add these for layer in model.layers: layer.trainable=False |
@zo7 Can you comment on i.e. why this code is not sufficient?
|
I am having a similar issue trying to pop the last layer of a ResNet50 model and add a new classifier. Using Keras version 2.2.0 with tensorflow backend. The previous fix in this thread for dealing with the outbound nodes and output is producing an error that I haven't seen mentioned yet. Is there a smarter way to pop the classification layer and add a new one, or a way to deal with this attribute error ?
This is the output
As an alternative, I know that the include_top=False option will remove the classifier (and the Flatten layer), but I don’t know how to correctly attach the output of this form of ResNet to a new classifier (e.g., using model.layers[-1].output ?). Any advice on this? |
pop() still does not work (Win 10, GPU Keras on CNTK backend, newest as of 2018-06-20). To be more specific, when a model is copied from a known model and is of type Sequential, it does not work. However, pop() will work on the original Model type model, so use it there to remove the last layer before creating a new Sequential type model from the original layers, i.e. vgg16. |
pls fix this problem as soon as possible, please i need this! |
https://stackoverflow.com/a/49403298/1179925 For some reason I need to build model with popped layer using
|
Sleep(1) |
Hi, model._layers.pop() |
I have a similar case(https://stackoverflow.com/questions/63202651/how-to-remove-layers-from-a-keras-model-in-order-to-use-as-baseline-for-creating) using model._layers.pop() as @amandeep25 posted seems to work (model.summary() returns desired result) however when checking tensorboard or exporting the model to .pb (final goal is to compile for deployment on raspberry) the original VGG16 model is still there. Any suggestions? |
@somewacko is there any way to pop a range of layers using indices? I mean without using loop or calling the same |
I believe model.layers = model.layers[:-3] # everything but the last 3 layers |
I'm trying to start https://github.com/iamaaditya/VQA_Demo which uses pre-trained VGG16 model (https://github.com/iamaaditya/VQA_Demo/blob/master/models/CNN/VGG.py). The last 2 layers are removed with layers.pop(). This doesn't seem to work, however. The message "ValueError: could not broadcast input array from shape (1000) into shape (4096)" is displayed, and I see the layers were not removed when I do "plot(image_model, to_file='model_vgg.png', show_shapes=True)". Thank you.
The text was updated successfully, but these errors were encountered: