Hi @danield , welcome to the forum!

This is an interesting case.

My instinct tells me Kendra might not know that the Box private key is encrypted. But this hans’t changed in over 2 year, so by it self it could not explain that it worked before.


But just in case could you check if Kendra has a place to put the passphrase and use an encrypted private key?

If not use openssl to decrypt the private key with the passphrase and store the unencrypted private key in Kendra.

I’m assuming the end goal is to use the private key to generate a JWT token…

Let us know.

Hi @rbarbosa,

Yes, PK generated by Box is encrypted. I created Box app with Server authentication with PK (tested both App Only and Enterprise scope). On Kendra side, I provide PK and passphrase (generated by Box) using AWS Secrets Manager, so Kendra connects to AWS Secrets Manager and retrieves that information from so-called secret. This secret is defined as follows:

This is the standard way to configure Box connector in Kendra. From the logs on Kendra side, I can see that Kendra attempts to connect to Box App and fails with the mentioned error. If I change any value in PK or passphrase, Kendra will report : “Invalid private key error”, so this is sort of confirmation that the format of PK is somehow valid.

In October 2023, I did a similar integration test a few times and authentication worked. Now, in Jan 2024 I received this error.

The client written using Box SDK and the same authentication information generated by Kendra works - as I mentioned - and this SDK is the same as I used in Oct 2023.

First, I would like to know which service fails. Do you think this error is something Box App sends back to Kendra?

Thank you !

Hi @danield

Well there are 3 steps here:

Kendra prepares a JWT token using the private key to build and assertion. There are a few details here let’s assume for now that those are correct.

Next Kendra requests an access token using the JWT token. Box API responds with an access token valid for 60 minutes.

Next Kendra uses the access token to hit the API end points.

A parsing error would occur on the first step, and if so then Kendra is not doing any requests to Box API.

If the error is on the second or third step then Kendra would get a 401.

Let’s explore a bit the private key format.

When you say with or without \n, was the final format you pasted something like this?

-----BEGIN ENCRYPTED PRIVATE KEY-----


MIIFHDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQIhVU1bLSvgl8CAggA


MAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECJ0m3yAKcUwpBIIEyKXKY3BVdPEU


oV0Ggy7XwLAyOTVbeiH+9amokZipHJdsvDT8DMDIawZsi/SATmWPqniJZjJgv0NC


<MANY MORE LINES>


zudVAvL8HvCH04djyWzVIdS0zC00bg086rWajDO2hqWJ2aNVNnbJX6uEM67wmCJW


PXID7YcZsgCzzK0vg/WvFSNXqot0tsWhErH4tTezi6SD069UiV6Uuvb210ec6RWc


6azMGCHf0q2skks79+e6UQ==


-----END ENCRYPTED PRIVATE KEY-----

Meaning the private key is broken down by lines, not a continuous line, by replacing the \n with a proper new line.

If not please do try. Also when you say paste as plain text, I’m not sure you mean the paste option like in a right click of the mouse or the plain text tab that shows up on your screen shot.

If this JSON configuration works on an Box SDK call, then it is highly likely the issue is on Kendra side. However you can open a support case with Box and ask for their help to see if any hits to the API are being made using your Box app client id.

Let us know if this helps.

Cheers

Hi @danield

I’ve just tested this with my app in Kendra and was able to start a sync.

I did replace the private key \n with new lines.

Hi @rbarbosa,

Thank you very much for your efforts. I know it takes some time to create a test environment for BOX and Kendra. I highly appreciate that.

In my case, Kendra scan starts as well, but fails after about 10min with mentioned error. Did your Kendra scan succeed?

I am getting this error in Kendra (latest).

FYI, Kendra scan usually takes 8-10 min if Box contains relatively small number of documents (10s-100s) but may take 3-4 hrs if there are more documents. I test Box - Kendra with about 20 files and the scan takes roughly 10min.

Hi @rbarbosa,

(I can not paste more than one screenshot, so I split my message into three)

About passing PK to Kendra:

This is original JSON generated by Box App (example):

{


  "boxAppSettings": {


    "clientID": "2rlrtcz5t2htyxp3nwsf3dvpr10eixxu",


    "clientSecret": "UleXS40pQ5xZrAa3FpUGS890BudO12Ws",


    "appAuth": {


      "publicKeyID": "6ytqadc3",


      "privateKey": "-----BEGIN ENCRYPTED PRIVATE KEY-----\nMIIFHDBOBgkqhkiG9w0BBQ0wUTApBgkqhkiG9w0BBQwwHAQIhVU1bLSvgl8CAggA\nMAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECJ0m3yAKcUwpBIIEyKXKY3BVdPEU\noV0Ggy7XwLAyOTVbeiH+9amokZipHJdsvDT8DMDIawZsi/SOPmWPqniJZjJgv0NC\n4Ao6xX5xRdqWbEUIw/ALfGje0m2XVu1ciZwApwDCIAR+ftd8DGEERJlGgV+7nqnm\nCTjuvXfj/rVBopLpL1XJ+I9esAvckLGc3t5UkR9NY0vHzDqsJ72Gsq1wj+uR2DTl\n8P2rwFDhhdhbQyWKr01FHmvNIsP1rHvLhG6/wcpiG+8CGcq/OX3vyZuUSO0rFPIL\n48iZvfh8bLwVllKuDSpdnj7xHlRF9KRmsgnSQ7wD1Rf5sshoMdltQMEp2nh7qstx\nG3tCSaR2tP8mf8MT5fUzTBCpjZH7rGt6YANgxIUZgwCAMMPfRxHc2zsCF+mtoSHp\nBivRSkeh7gEeA0r/OTp0RutcudejCgVJZB3EG4P+R0JDZ1Sz4Q7Vd6muSZdiSuuj\ntFiQKhTEO9J1bwr4zkYcaF4oMABTN1ak7kSdtaRlGZLyycfbU1BOzlrR9gTztRHH\nz64GvWU14BnJARp2Ub4AvOtcRqnOU6iLraf/zpy5R+zJwHXYVZJ/l7x5Qy87sSx4\n/LHTxNONZk0j5H8g/0CQEsvy1mMA85qh+Xf/itYPE3Icrm43CMK2E4yfrUbNdw4X\nvt7WUWQjDPNdaSFTiLmYLgxritki0lHu4TMGhFSJ+mkzx/EtBqDxTdpYqA5TxB0R\nuNTZ+lASgNIDUJLtMf5Jj0YAxAA+VxCXIxbR+OVWB59V1NCPmUcsixHqINKQXb0r\nx1ehZMME7xgYbR1GGuSIWF2RxiXUovqMQCZD3aht7yQovOvElaZJ/6B0T87TzXtY\n+wYeVNShWIBA55NIWbMvsV4Eo0EcQTt419A38iBT4Ds5LI3SXHQHIbOOK4i/cGuW\nXm2Zk2l9waeSk5ZSNVxiurUsaxsv6bqJD96Q5Sc3AFepL8+D4VuCCzeJ3f3oBeMo\nXTQ1QXI2GaDAdI6uSwnj4FsWD9hjztv5b6cipujvz3yJGvZscLGwlKOv9ozb8mUB\nyYYecMZ72CVQgdHFPjOIE2qUKgTXEJO80eZAebSDZ0SlyJHTCDUqTCbZ0kz5Nu7v\nILd041HfnqfyPItz5HR58KHjHduyFHnFFOupKbz2tcAJ9Vm1lf8YAiKFN57V7J1N\nuFIRqSf/yVR64W2V2vUl55KUt8jE5dGKrKZifUCY1W52LUr6r+scFdcYG0Joeb1i\nduKjYtXWWKVrgVcYeYAAD+VmgJ/VrbnuXrraXViibqzg6ScELNh6eJGrEHtbC5NB\ntSzRsPQvni1yuWlvnLaCG8QVA1RKlKYTUjR+X1NQGngzqpiCVgeEXAoMn/Mna4g0\nbISqY/RXZ9V/H3Ux/UoQmqktjIHdkfo18dpP7rtsLNvUobkdLF71Kgf67AosW4dq\nFChliTTkhr2L8Rwmfyq4PUQeov6QCfSMROa903woZJO75gJiFh1VcJyUV0+DWSLg\nzudVAvL8HvCH04djyWzVIdS0zC00bg086rWajDO2hqWJ2aNVNnbJX6uEM67wmCJW\nPXID7YcZsgCzzK0vg/WvFSNXqot0tsWhErH4tTezi6SD069yBV6Uuvb210ec6RWc\n6azMGCHf0q2skks79+e6UQ==\n-----END ENCRYPTED PRIVATE KEY-----\n"

In AWS Secrets Manager secret I can define keys and values called respectively “secret key” and “secret value”. Each “secret key” field has “secret value” field., like this:

When I fill “secret value” field for AWS Secrets Manager secret, I did it in the following ways:

1. I copied the contents of “privateKey” as is into the secret (so this string includes “\n”)



2. I copied the contents of “privateKey” removing “\n” (long continous string)



3. I copied the contents of “privateKey” replacing “\n” with space

Kendra failed using the above methods,

Hi @rbarbosa ,

Instead of entering the string into “secret value” field, I can switch to “Plaintext” and copy JSON files contents (generated from BOX) into this “Plaintext” field. Then the formatting will be like this ;

In all the above cases, the Kendra scan ends with “Can not parse PCKS…” error.

Just to repeat my question, did your Kendra scan succeed?

Hi @danield ,

It did complete with some errors, but nothing like the parsing of the private key. It was able to index some files.

For example:

{


    "LogLevel": "Info",


    "AwsAccountId": "533267234712",


    "IndexId": "0a226dc0-44ae-4ab3-9aab-bc3bf5945ac9",


    "SourceId": "93e533f4-1142-475b-b66f-a7f20e19abbc",


    "DocumentId": "974228631587",


    "DocumentTitle": "Document (Powerpoint).pptx",


    "Message": "Indexing document to index."


}

Some error examples:

{


    "LogLevel": "Error",


    "IndexId": "0a226dc0-44ae-4ab3-9aab-bc3bf5945ac9",


    "DataSourceId": "93e533f4-1142-475b-b66f-a7f20e19abbc",


    "DataSourceSyncExecutionId": "4af11dd3-a16b-461a-be27-48f932c37fe6",


    "DocumentId": "1243040974444",


    "ErrorCode": "InternalError",


    "ErrorMessage": "File doesn't contain any data"


}

{


    "LogLevel": "Error",


    "AwsAccountId": "533267234712",


    "IndexId": "0a226dc0-44ae-4ab3-9aab-bc3bf5945ac9",


    "SourceId": "93e533f4-1142-475b-b66f-a7f20e19abbc",


    "Message": "[aeb40c07-8f84-4352-96f0-e1aee17f4ca8] - Custom metadata not found. 1127009206585"


}

{


    "DocumentId": "1410588695626",


    "IndexName": "box-index",


    "IndexID": "0a226dc0-44ae-4ab3-9aab-bc3bf5945ac9",


    "SourceURI": "https://app.box.com/file/1410588695626",


    "IndexingStatus": "DocumentFailedToIndex",


    "ErrorMessage": "Document cannot be indexed since it contains no text to index and search on. Document must contain some text",


    "ErrorCode": "400"


}

So from an authorization perspective it did work.

I did replace the \n with a new line, so my secret does look like yours:

Now, l want to discuss your private key, but I’ll use a private message.

Hi @rbarbosa,

I received your private message. Unfortunately, I don’t know how to send private messages.

The private key example is not valid (the one you tested), just random characters to illustrate the format. Thank you for providing openssl command to verify private key. I did the test on my side and I could decrypt currently used private key with currently used passphrase, so this key I am using now is valid for sure (and works when using Box SDK),

This test - however - has caught my attention and I found something what looks like the cause of the problem. Kendra uses AWS managed key (stored in KMS service) to decode the secret stored in Secrets Manager and the key id is provided in the IAM role for Kendra.

The keys managed by AWS are periodically rotated in Secrets Manager and I found that we were using the key generated by AWS in Oct 2023 - meaning this is an invalid key, so even though I provided the private key information to Kendra properly, Kendra was not able to retrieve it. I updated IAM role for Kendra allowing Kendra accessing the key with current KMS key id.

I re-tested the connectivity, However, the connectivity from Kenra to Box App still fails. Furthermore, I copied private key into secret’s fields as is in this test. Now I am trying to provide the private key in Secrets Manager in such a way that the displayed format will be as above (multiple lines with real line breaks).

I think you did exactly the same, However here I have a question. How did you exactly enter private key characters into Secrets Manager? Did you use “plaintext” tab in secret and copy private key in JSON format (so JSON will be automatically converted to multi-line format)?

I will report back on the test result.


Daniel

You’re making progress, awesome!

That makes sense and explains why it stopped working. Since I was creating the datasource from scratch it never got to rotate the pass phrase.

To enter the Secret value, I used the Key/Value input, not the plain text. Actually the plain text option does not show when I was creating the datasource and got to the private key section.

To format the private key I copied the long string into visual studio code, then find \n and replace by newline (Shift + enter) in macOS.

I’ll exemplify with the fake private key:

Click replace all and save:

Cheers

Hi @rbarbosa,

Right. My current environment is fully new, except for IAM role for Kendra, which I reused …and the key has changed in the meantime.

I formatted the private key in exactly the same way as you have done, and I also added the formatted key to key/value field, but the old KMS key (before rotation) has been used for the test and the test failed.

Now I am going to test the combination: newest KMS key with “\n” → new line conversion (for private key) in the key/value field (Secrets Manager). Hopefully this will work.

Hi @rbarbosa,

Even though I have used the current KMS key and replaced “\n” with new line in PK, the test failed with the same error. I can only think that this is due to how Windows encodes new line character which is CRLF (and I use Windows). In your case, new line character will be probably be LF instead of CRLF (since you use Mac), so I will have to do yet another test.

Hi @rbarbosa,

Unfortunately, I lost access to my test environment and had no chance to do the last test, my BOX account two weeks trial has expired. I am left only with AWS environment. I have to put on hold this testing.

I don’t know if I can ask, but would it be possible to connect to your Box App from my AWS account ?

Hi @danield ,

It is not possible to connect using my account, sorry.

But wouldn’t a free account work for your test?

Cheers

Hi @rbarbosa,

My apology for late reply. About this issue I received communication from AWS that it was Kendra’s bug preventing me to connect to BOX App. The bug is now fixed by AWS and I can connect to BOX App. Thank you for your support, it helped me to narrow down the issue with Kendra.

However, I have yet another problem. I would like to scan documents in Box using App User created in Box App. This App User has permission to access limited number of Enterprise files. When Kendra performs scan with App Only user scope, in Kendra’s log I can see that limited set of files is detected (this is correct), but the files are not added to Kendra’s index. Do you know maybe if indexing via App User is supported by Box/Kendra?

(Should I open new topic for this issue?)


Best regards,


Daniel

Hi @danield

That is strange, and I’m not sure what we can do on the box side, since it should add those files to the index.

Just to eliminate all possibilities, if you create a simple download script, or using Postman, can you download those files when using Kendra’s security context (the APP user?)