Get out a meter and check continuity with a stereo cable. Draw a diagram and solder together.
If you google around, you may conclude the Raspberry Pi does have some noisy analog audio, but that's not the case in every setup. The culprit is due to a ground loop issue. You can do a few things to alleviate the issue:
According to Porta Pi users out in the field and my testing, purchasing a ground loop isolator is the way the go. (You can also make a few for about $5. Read on for the DIY solution). A ground loop isolator eliminates the buzz by about 80-90%. These devices also work other places, like your MP3 player (or phone) plugged into your car stereo. (In this car example, if your MP3 player is also powered by the car you might hear that buzzz. If the MP3 player is running off its own battery, then you likely won't hear that buzz).
Plug in that group loop isolater and done.
You should also turn the alsamixer volume up higher (to about 80% or more). This boosts the pure audio over the noise.
How do I diagnose the issue?
If you have positive voltage connected to your amplifier (with the speakers connected and nothing else), try connecting the audio cable from the RPi to the amp. If the amp powers on, then the power loop found a ground path through the analog audio ground path. If you connect the correct ground (to complete the loop powering the amp) you'll hear the noisy, hisssy, buzzzzzy audio even more prominently. We introduced the proper ground and subsequently a group loop.
What causes this issue?
Unlike moving audio digitally like through HDMI, the analog audio path we've wired offers more chances for contamination between the two grounds in the arcade: the ground we use for powering everything and the ground reference used in the analog audio signal. Remember: the audio amplifier needs power and ground. The audio signal also needs a ground reference. If these two 'grounds' find their way to each other we have a ground loop with possible different voltage potentials (albeit very small) on each ground. The audio signal becomes coupled with the same potential used in the power plain and we start hearing the beeps, boops, crackles and pops of the power side coupled into the audio side.
How do we fix it?
We need to separate the audio and power grounds. A simple way to do this is power your analog amplifier with a battery pack. This makes the audio signal ground separate from the
audio power ground. But that's not practical- replacing batteries for a unit that's plugged in anyway. We need a different way to make these two grounds from flowing into each other
We need to buy (or make) A ground loop isolator. Adding a ground loop isolator, simply puts a transformer in the middle of the analog audio signal that will prevent the ground loop.
Let's build on. In this rare case, building one is cheaper thank buying one. Best I could find was $8 for a consumer ground loop isolator for small audio setups. We should be able to build one for under $5 (we will need to buy more parts than we need to get those sweet bulk rates on ebay).
First, and my favorite: off to ebay to buy some super cheap stuff and hurry-up-and-wait 10 to 25 days for its arrival. I grabbed some audio transformers and PCB mount 3mm stereo jacks. I checked digikey for audio transformers and oh my, ebay wins by 1/10 of the price.
I made a PCB for this. Email and I will send you one.