We spent the 2 first nights and the last one in Cayenne. Airport is 15km far from downtown. As we had no reservation, our choice was the "Central Hotel", in Cayenne downtown, and one of the cheapest there. Another suggestion is "Grillardin" hotel, 3km far from airport and cheaper. For your travelling, don't hesitate to rent a car directly at the airport, there are not a lot of roads in French Guyana and you can't be mistaken !
There is no activity in Cayenne on Sunday : all is closed, no traffic at all. As we arrived on Saturday night, we had to wait till Monday to do our shopping (water, cakes and other grub, wire netting for traps, plastic jars, batteries, oil, gas can for the generator, …). All shops are managed by asian people. TIP 1: do shopping before leaving Cayenne, because there is no way to buy something on Kaw road nor on Regina road (except in Cacao). TIP 2 : little chinese shops are cheaper than bigger shopping centers.
In Cayenne, you can visit on foot ("place des Palmistes", the beach, "General de Gaule" road, …). There is an entomological shop in Cayenne, but there was not a lot of specimens for sale when I was there : L'Arlequin. You can also find big spiders in all souvenir shops. By the way, when shops are open, you can find anything in Cayenne !