Is there a cost to cancel an order?

Yes, cancellation requires an onchain transaction, so you’ll pay a small gas fee. There’s no protocol cancellation fee.