encrypt with public key in android

Question

I have an assignment from school for making an application with RSA functionality. I read a lot of documentation of how I should do it and such and most of it is working, except one thing I can't wrap my head around. The thing I need/want to do is extract the modulus and exponent of the public key, send it the database so that it can be shared with different users (like in a chat application).

This is all working fine, I can extract, and I can generate the public key with these values, the problem I do face though, is that it generates a key as "openSSLRSAPublicKey". (See picture):

Wrong RSA keys

The picture shows my public key in the AndroidKeyStore on the phone, and the "friend public key" which is converted already.

I'll include some code snippets as well as to show how I do these things, if anything should be unclear ask and I'll try to update my question.

This is how I generate my keys:

// if keypair alias doesn't exist yet make a new pair, and return the public key
// if exists already return the public key
public static PublicKey createKeyPairsOrGetPublicKey(String Uid) {
    PublicKey publicKey;

    try {
        KeyStore keyStore = KeyStore.getInstance(ANDROID_KEYSTORE);
        keyStore.load(null);

        if (!keyStore.containsAlias(Uid)) {
            KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance(KeyProperties.KEY_ALGORITHM_RSA, "AndroidKeyStore");
            keyPairGenerator.initialize(
                    new KeyGenParameterSpec.Builder(
                            Uid,
                            KeyProperties.PURPOSE_DECRYPT
                                    | KeyProperties.PURPOSE_ENCRYPT
                                    | KeyProperties.PURPOSE_VERIFY
                                    | KeyProperties.PURPOSE_SIGN)
                            .setDigests(KeyProperties.DIGEST_SHA256)
                            .setSignaturePaddings(KeyProperties.SIGNATURE_PADDING_RSA_PKCS1)
                            .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_RSA_PKCS1)
                            .build());

            KeyPair keyPair = keyPairGenerator.generateKeyPair();

            publicKey = keyPair.getPublic();
        } else {
            KeyStore.PrivateKeyEntry privateKeyEntry = (KeyStore.PrivateKeyEntry)
                    keyStore.getEntry(Uid, null);
            publicKey = privateKeyEntry.getCertificate().getPublicKey();
        }

    } catch (KeyStoreException | CertificateException | NoSuchAlgorithmException | IOException | InvalidAlgorithmParameterException | NoSuchProviderException | UnrecoverableEntryException e) {
        e.printStackTrace();
        publicKey = null;
    }

    return publicKey;
}

This in turn extracts the modulus and exponent

// createKeyPairsOrGetPublicKey is a method that's shown as first in this question

PublicKey publicKey = Helper_Security.createKeyPairsOrGetPublicKey(user.getUid());

if (publicKey != null) {
  RSAPublicKey rsaKey = (RSAPublicKey) publicKey;
  BigInteger publicExponent = rsaKey.getPublicExponent();
  BigInteger publicModulus = rsaKey.getModulus();

// the hashmap is used to put in values and send it to firebase.
  HashMap<String, String> userMap = new HashMap<>();
  userMap.put(FIRST_NAME, firstNameInputEt.getText().toString());
  userMap.put(LAST_NAME, lastNameInputEt.getText().toString());
  userMap.put(PUBLIC_EXPONENT, publicExponent.toString());
  userMap.put(PUBLIC_MODULUS, publicModulus.toString());

    myRef.setValue(userMap).addOnSuccessListener(new OnSuccessListener<Void>() {
      @Override
      public void onSuccess(Void aVoid) {
        // When the send is a success open another activity
        Intent intent = new Intent(
        parentActivity, MainActivity.class);
        startActivity(intent);
        parentActivity.finish();
        }
    });
  } else {
  Toast.makeText(parentActivity, "Something went wrong", Toast.LENGTH_SHORT).show();
}

next I convert the modulus and exponent back to a public key

// convert the public key from the friend from a string to a public key
public static RSAPublicKey convertStringToPublicKey(String publicExponent, String publicModulus) {
    RSAPublicKey publicKey;

    try {
        BigInteger modulus = new BigInteger(publicModulus);
        BigInteger exponent = new BigInteger(publicExponent);
        RSAPublicKeySpec rsaPublicKeySpec = new RSAPublicKeySpec(modulus, exponent);

        KeyFactory keyFactory = KeyFactory.getInstance("RSA");
        publicKey = (RSAPublicKey) keyFactory.generatePublic(rsaPublicKeySpec);
    } catch (Exception e) {
        e.printStackTrace();
        publicKey = null;
    }
    return publicKey;
}

after this all, I want to encrypt a message with the public key:

// encrypt a text with a public key
public static String encryptWithPublicKey(PublicKey publicKey, String textToEncrypt) {
    try {
        Cipher inCipher = Cipher.getInstance("RSA/ECB/PKCS1Padding", "AndroidKeyStoreBCWorkaround");
        inCipher.init(Cipher.ENCRYPT_MODE, publicKey);

        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
        CipherOutputStream cipherOutputStream = new CipherOutputStream(
                outputStream, inCipher);
        cipherOutputStream.write(Base64.decode(textToEncrypt, Base64.DEFAULT));
        cipherOutputStream.close();

        byte[] vals = outputStream.toByteArray();
        return Base64.encodeToString(vals, Base64.DEFAULT);
    } catch (Exception e) {
        e.printStackTrace();
        return null;
    }
}

This in turn gives me an exception:

    W/System.err: java.security.InvalidKeyException: Unsupported key type: OpenSSLRSAPublicKey{modulus=d695704c74b3392c48574a62cb8503fb5204998e41d434199df75aa81813e66e263e32e537fcdc2924287c7b817aee39d34c933145d131ac86e40d31752064d86a782f5384da54d7f18d105b85dc3e6dc9e0adf9614e697cf7898fb83c97d37768a89674a7240defe2dbe45cad70aa9a1d753b7f6658a9cba018ee7e89f720d358f4788055f72116dbd041cc3adcf7e97350a67d0c6fbc926561547e3ad30548ea0abccea68d701b04d26aa7fc4fca40b6bedeb2c4dd0c94f19ad06b60c39ac57fea05106e497b5fe9163bd3f6d06ef0fd8934cd933f2bb8b328d04c719ca7a5b300c5d0214a5d46b406171c2a05c5da8103a361bff6e88da7f557e261f62ed5,publicExponent=10001}
    W/System.err:     at android.security.keystore.AndroidKeyStoreRSACipherSpi.initKey(AndroidKeyStoreRSACipherSpi.java:371)
    W/System.err:     at android.security.keystore.AndroidKeyStoreCipherSpiBase.init(AndroidKeyStoreCipherSpiBase.java:169)
    W/System.err:     at android.security.keystore.AndroidKeyStoreCipherSpiBase.engineInit(AndroidKeyStoreCipherSpiBase.java:105)
    W/System.err:     at javax.crypto.Cipher.tryTransformWithProvider(Cipher.java:612)
    W/System.err:     at javax.crypto.Cipher.tryCombinations(Cipher.java:521)
    W/System.err:     at javax.crypto.Cipher.getSpi(Cipher.java:437)
    W/System.err:     at javax.crypto.Cipher.init(Cipher.java:815)
    W/System.err:     at javax.crypto.Cipher.init(Cipher.java:774)
    W/System.err:     at com.pxlpe.chatapppe.Helpers.Helper_Security.encryptWithPublicKey(Helper_Security.java:102)
    W/System.err:     at com.pxlpe.chatapppe.fragments.ConversationFragment.onViewClicked(ConversationFragment.java:104)
    W/System.err:     at com.pxlpe.chatapppe.fragments.ConversationFragment_ViewBinding$1.doClick(ConversationFragment_ViewBinding.java:37)
    W/System.err:     at butterknife.internal.DebouncingOnClickListener.onClick(DebouncingOnClickListener.java:22)
    W/System.err:     at android.view.View.performClick(View.java:5697)
    W/System.err:     at android.view.View$PerformClick.run(View.java:22526)
    W/System.err:     at android.os.Handler.handleCallback(Handler.java:739)
    W/System.err:     at android.os.Handler.dispatchMessage(Handler.java:95)
    W/System.err:     at android.os.Looper.loop(Looper.java:158)
    W/System.err:     at android.app.ActivityThread.main(ActivityThread.java:7224)
    W/System.err:     at java.lang.reflect.Method.invoke(Native Method)
    W/System.err:     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1230)
    W/System.err:     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1120)

My thought is that I somehow need to convert the openSSLRSAPublicKey to an AndroidKeyStoreRSAPublicKey, but I can't find any documentation what is the real problem, I looked at the byte code and that is exactly the same, when debugging is see the modulus and exponent are correctly used as well.

Any pointers are highly appreciated, thank you in advanced and happy coding!!

Answer 1

The problem was exactly like @Maarten Bodewes in comments pointed out. I deleted the "AndroidKeyStoreBCWorkaround" and everything works correctly now.

 // encrypt a text with a public key
    public static String encryptWithPublicKey(PublicKey publicKey, String textToEncrypt) {
        try {
            // changed this:
            // Cipher inCipher = Cipher.getInstance("RSA/ECB/PKCS1Padding", "AndroidKeyStoreBCWorkaround");
            // to this and all seem to work now
            Cipher inCipher = Cipher.getInstance("RSA/ECB/PKCS1Padding");
            inCipher.init(Cipher.ENCRYPT_MODE, publicKey);

            ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
            CipherOutputStream cipherOutputStream = new CipherOutputStream(
                    outputStream, inCipher);
            cipherOutputStream.write(Base64.decode(textToEncrypt, Base64.DEFAULT));
            cipherOutputStream.close();

            byte[] vals = outputStream.toByteArray();
            return Base64.encodeToString(vals, Base64.DEFAULT);
        } catch (Exception e) {
            e.printStackTrace();
            return null;
        }
    }