My condolences on the loss of your brother-in-law.
I'm always a little afraid to ask this question because typically the answer given is "we didn't know about this".... Did your BIL donate his brain upon death such that the LBD diagnosis could be confirmed? That's the *only* way to know if he had LBD. A more common dementia with "severe behavioral problems" in the middle-aged is FTD (frontotemporal dementia). LBD is often misdiagnosed.
I think if your BIL has siblings or parents with dementing illnesses, then there's reason for concern for his children. But if there's no other sign of dementia in the family, then there's less concern for something inherited.
In PD and AD, if the illness is early-onset, there's much higher likelihood of the patient having inherited the disorder. I'm not aware of any studies in LBD that would say whether this is true in LBD.
In our local support group, we have two people in their 50s with a clinical DX of LBD.
Your young relatives are welcome to join us here. I'm sure there's also an online grief group (though not specific to LBD) that they could join. I'd encourage the family to consider counseling for the children.
Finally, you might dig around the Well Spouse site's Kids Forum to see if anything there is appropriate for your young relatives: