Based on Alessandro Baricco´s novel the movie tells the story of happily married Herve Joncour (Michael Pitt), a silk breeder from the south of France. When, in1861, silk production in his home town comes to a near stand-still due to a disease that kills the entire silkworm population, a local silk magnate (Alfred Molina) finds a solution: the import of disease-free silkworms from Japan. He enlists Herve to make the long and lonely trip to the Japanese inland in order to aqire and smuggle out a load of the precious cargo. Once there, he encounters a beautiful young woman "who does not have oriental eyes" (played by Sei Ashina) and , despite not exchanging a single word with her, falls head over heels in love with her.
Unfortunately, she happens to be the concubine of his local trading partner. Despite the danger, as Japan threatens to explode in civil war, and despite his marriage to the beautiful and loving but barren Helene (Keira Knightley) he continues to make these business trips, each time laying his eyes on the woman to whom he has become enthralled. Although they have no common language and therefore are unable to exchange a single mutually understandable word, Herve and the concubine share a secret love.
Upon receiving a note from her in which she reveals her love for him, his home-life starts to unravel and he becomes more and more obsessed with the woman at the other end of the world. He channels his longing and his frustration by planning and building a park around his house and taking his wife on holidays throughout Europe.
When a letter arrives from Japan he is devastated by its content. While professing eternal love and devotion the woman warns Herve never, never to try to contact her again...