46 Ford Coupe Groans upon deceleration

[Jacobs1946]

Hoping someone out there might be able to help me diagnose this noise?!?
I have a '46 Coupe with the V8. All original down to the 6 volt system.
Runs awesome.
When accelerating, it sounds perfect. When you let off the gas, there is a groaning noise coming from the transmission area.
The person I bought it from said it was a universal joint that was worn.
I changed that and did find there was no bolt and washer holding the U Joint to the transmission output shaft. I bought the special bolt and washer from CW Moss, and installed it.
I've changed all motor mounts, new clutch, pressure plate, throwout bearing.
When the motor and transmission was out, I checked the input and output shaft on the transmission. Seemed tight. The torque tube bearings seemed tight. The rear end doesn't seem to have any noise, (it's the original banjo type).
Also, when I disengage the clutch, the groaning noise is less loud or goes away completely.
Several things on google say it could be a worn pinon bearing, but the noise comes from under the front floor. Any ideas?