ag.algebraic geometry - Relatively ample line bundles

EDIT : my previous answer was wrong. Thanks to BConrad for pointing it out.

Here is a counterexample if the map is not proper. Let $X$ be the plane, let $Y$ be the blow-up of the plane in one point, and $U$ be $Y$ with one point of the exceptionnal divisor removed. Let $f|_U:Uto X$ be the projection and consider the line bundle $mathcal{O}_U$.

Since the fibers of $f|_U$ are affine (either points or the affine line), $mathcal{O}_U$ becomes ample when restricted to fibers of $f|_U$. However, $mathcal{O}_U$ is not $f|_U$-ample.
Indeed, if it were, $mathcal{O}_U$ would be ample on $U$, but we can compute :
$$H^0(U,mathcal{O}_U^N)=H^0(U,mathcal{O}_U)=H^0(Y,mathcal{O}_Y)=H^0(X,mathcal{O}_X),$$

where the second equality comes from property $S2$ and the third holds because $f_* mathcal{O}_Y=mathcal{O}_X$. Hence, $H^0(U,mathcal{O}_U^N)$ cannot distinguish between two points of the exceptionnal divisor, and $mathcal{O}_U$ cannot be ample on $U$.

---

Warning : what follows is false. I kept it here so that the comment below remains understandable.

Here is a counterexample if the map is not proper. Consider the inclusion $f$ of the plane minus a point $U$ in the plane $X$. The line bundle $mathcal{O}_U$ is ample restricted to the fibers of $f$ (they're points...). However, it is not $f$-ample. Indeed, if it were, $mathcal{O}_U$ would be ample on $U$. Choosing $N>>0$, we would get $$H^1(U,mathcal{O}_U)=H^1(U,mathcal{O}_U^N)=0.$$
But a simple computation via Cech cohomology shows that this cohomology group is not trivial (in fact, infinite).

---