The first use of an apostrophe was in the 1500s, used to indicate omissions. Think can't for can not, that's for that is, it's for it is, etc.
One of the more common mistakes happens with its and it's. Its (with no apostrophe) is a possessive pronoun, just like hers, ours, and yours. It's (with the apostrophe) is the contraction of it is, as noted above on usage to indicate omissions.
Many people struggle with making singular and plural words that end in s possessive. When a singular word ends in s, I'm inclined to add an apostrophe to the end, with no s. The Associated Press recommends this, and I tend to follow AP Style. With that, when talking about the restaurants of Kansas I would write Kansas' restaurants. When that plural word ends in s, the same rule applies. Just add the apostrophe at the end to make it possessive. If the plural word doesn't end in s, then you'll need the apostrophe s at the end. When you are trying to make a single letter plural, you'll need the apostrophe (Mind your p's and q's.). This is especially true when you're talking about a's, i's, and u's... otherwise your readers will think you're writing as, is, and us.
One of the apostrophe errors I see quite a bit is with plurals. Like Grammar Lady says above, "Apostrophes are for possessives, not for plurals!" This is not to be confused with what we just talked about above. Remember, we were making those words possessive. What I'm talking about is the sign that reads "Banana's For Sale", or "Ladie's Night", and "Happy Holidays from the Smith's". The basic rule for plurals is that they are formed by adding s to the singular form. Those signs should read "Bananas For Sale", "Ladies Night", and "Happy Holidays from the Smiths".
There's so much more that can be covered with the apostrophe, but I don't want your head to start spinning! This should cover some of the more common questions out there about apostrophe usage. As always, if you have any questions I would be happy to help clear them up! Just comment below or send me an email!