I've been having an email exchange with someone about "open" endings in books. They can be tricky for readers to accept, since human beings generally like two things in story endings: justice and resolution. But some readers are open to endings that are unhappy, or uncertain, or ambiguous. The world is in short supply of justice and resolution, and while many readers want those things in books as an antidote to that harshness, other readers hunger to see that truth reflected in their literature. They find comfort in the fact that literature can say: Yes, this is difficult; no, it's not fair or right. Or, I don't have all the answers either, but here is what I know to be true.
In the case of an an ambiguous ending, I think it works if the whole rest of the book supports that uncertainty, and if the reader has enough clues to go on with. I think any book should leave some unanswered questions, things for the reader to wonder about, even if all the main questions are answered. But where the main questions are not answered, then the readers should at least receive the tools to build a satisfactory ending for themselves--even if they all build different endings. (The power of being able to tailor the ending to their own ideas is part of the attraction of an ambiguous ending.)
An "open" ending shouldn't drop the reader off a cliff, and shouldn't be based on the writer flinging up her hands and saying, "I've run out of ideas; I don't know what happens next." An open ending should be as carefully planned as any other part of a book.