No that's not possible.
While setting the campaign, you will be able to choose Cashback by order or Cashback by Product. However, only one of them can be activated at a time to avoid overusing.
Yes, you can set the Credit amount as a percentage of the order/products or a fixed amount. Especially, with percentage, you can set the maximum amount to avoid big credit amount.
Koin will consider After-Discounted subtotal. Meanwhile, it will not calculate Shipping Fee and VAT.
For example, in 2 orders below, I set the Cashback by order as 10%. Koin will take into account the discounted subtotal: $337, and the credit amount for this order is $33.7. The VAT and Shipping Fee won't be added.