on the textbook? in the textbook?

in the textbook

日本語で「教科書のどこ」と言うから where を選んだのでしょうが、need help with (+noun) なので、what です。

What in the textbook do you need help with?

What section of the textbook do you need help with?

where を使うのなら、次のような言い方ができます。

Where is the problem in the textbook that you need help with?

on the textbook だと意味が違います。

My dog was sitting on the textbook. No wonder I couldn’t find it.