In my work I usually deal with Brownian motion and white noise so I am not the expert on the topic of fractional Brownian motion (fbm). But I recalled that I have seen an engineering application in the David Nualart paper on fbm (Section 4 - fbm in turbulence) . Sorry, it is not on signal processing and it has a finance part (Section 5) but it is a good short intro on the topic of fbm. Also interesting paper for you could be this (fbm in a nutshell) . You could check out the references in both of this papers additionaly. But, if you type "fractional brownian motion" into the Science direct you get a lot of books that mention signal processing in engineering (fbm in engineering) . And this would probably be your best shot in finding an answers to your questions. Hope this helps you somehow.