Ministers must send clear signals that they believe in new forms of green technology if they want firms to invest in them, a think tank says.