The short answer
If you sell physical products through Amazon, yes — and it is not optional, because competitors bid on your category page and organic placement will not hold position.
If you are a service business, a practice, a contractor, or a restaurant: no. There is nothing to buy and nothing to convert.
The two edge cases
Businesses with a product line alongside a service — a pool company selling chemicals, a salon selling its own products. Amazon can be a genuine second revenue stream, but treat it as a separate business with its own economics rather than as marketing for the service.
Businesses in categories where Amazon now sells adjacent services and is capturing searches you used to own. Worth knowing about, not usually worth advertising against.
What Amazon advertising actually is
Sponsored product and brand placements inside a purchase environment, priced by auction, with conversion measured in the same session. It is the most closed-loop advertising available and its returns are usually clear within weeks.
That closed loop is exactly why it does not transfer to a local service business. There is no basket.
The general lesson
Every year brings a channel with real momentum and impressive case studies from businesses that are not yours. The filter is always the same: does my customer buy this way, in this place, at this moment?
If the answer is no, growth in that channel is genuinely irrelevant to you, no matter how large it gets.