One theory is that the phrase evolved from the similar, but less commonly used phrase, "out in left field," which means out of it, absurd, or nonsensical.
Idiom: Out of left field
Definition: Unexpected, surprising, coming from an unexpected place
Example: "Your suggestion to solve the fiscal crisis by filling mailboxes with water really came out of left field."
Origin: Although obviously a phrase relating to baseball, the actual origin of "out of left field" remains mysterious. As usual, there are many theories.
The odd thing about the phrase is that in baseball, things generally don't come out of left field, unless a ball is hit into left field and is thrown infield by the left fielder, in which case it would not be at all unexpected or surprising, and thus wouldn't explain the origin of the idiom.
One theory is that for a runner rounding third, a ball thrown to the catcher from left field would appear to be coming from out of nowhere, but that seems like a bit of a stretch.
Some linguists think the phrase evolved from the similar, but less commonly used phrase, "out in left field," which means out of it, absurd, or nonsensical. But even the origin of this phrase is not clear. Some have theorized that it originally referred to a mental hospital that was located behind left field near Chicago's old West Side Park.
Still another theory is that anyone who had a seat behind left field in Yankee Stadium when Babe Ruth played was out of it because the Babe played right field.
Other theories relate the phrase to the now obsolete notion that anything "left" was evil. The left had was the Devil's hand, and one's left side was known as one's "sinister" side, due to the fact that the Latin word "sinistra" meant both "left" and "unlucky."