I'm using a basket strainer as a hopback from kettle pump to heat exchanger. It's got 3mm perforations. I fill it with whole hops though. Wouldn't think of using pellets, but a useful adaptation for me is to use a fine mesh nylon bag in it as a bag filter for CIP the heat exchanger. Takes out the particulate during the CIP as well as being a CIP solution reservoir. The bag needs a structure to keep in intact, otherwise the hydraulic pressure would just rip it apart. Think of your coffee maker without a structure to hold the paper filter. Basket strainers are not expensive and would suit your application fine. Be aware that pellets are ground very fine and will tend to form an impenetrable sludge quickly. Why wouldn't you use the (arguably) finer characteristics of a whole hop? Much easier to filter through.
Palau Brewing Company