had eaten も had been eaten も、どちらも過去完了形の時制になっています。
この時制を現在形に戻すと、
eat と be eaten になります。
つまり、能動態か受動態かの違い、ということになります。
The frog had eaten the insect before I was able to separate the two.
The insect had been eaten by the frog before I was able to separate the two.