Secret key should be passed as Uint8Array. nacl will give you the keys in the correct format already.
You can also do the following:
const secretKey = TonWeb.utils.hexToBytes('YOUR HEX');
const keyPair = TonWeb.utils.nacl.sign.keyPair.fromSecretKey(secretKey);
On the other side, you can check the detail here like in Jetton Example by using Tact Language