Well, everyone's children will be different and we've had different struggles with some of our little ones than others. Right now our children are two girls age 11 and 6 and FIVE boys age 7, 4, 3, 1, and 7 months. We are majorly outnumbered by little ones so training them to be relatively quiet and still is essential. If one spouse takes out a little one the other is left with 6 children and it could get bad quickly without some training ...LOL. Church is 3 hours long on Sunday, with the first 70 minutes with all members in one room on pews. There is no nursery available during this period of time. It is expected that we worship as families.
What has worked best for us:
No toys beyond something for the baby to chew on. Period. They're just a distraction and something to fight over.
Plain paper and something to take notes/draw with.
Pretzels or some other dry, small snack. This is passed out halfway through Sacrament meeting (45 minutes into the meeting).
No leaving the room for bathroom breaks or drinks. You go before it begins or don't go.
We try to sit behind a family that does not bring toys or electronics (seriously, a few families use Ipods/Ipads/Iphones to keep kids quiet during services - one family has them for all 4 kids up through the teen. They play games, text, and surf the web). These families are a distraction.
We expect our children to sit on our lap or the pew, not wander back and forth on the floor or lay under the pew, or stand on the pew. No laying either. With little ones we hold them or they sit beside an adult. We shush them quietly, redirecting their eyes to the speaker. During hymns we hand them a hymnal and help them follow along with a finger. If they get loud or upset and won't quiet we take them out to a small classroom. We sit on a chair in the middle of the room without turning on the light, with the little one on our lap. (Bigger ones get their own chair). They may not get up, play, or do anything fun. When they are ready to be quiet we go back and sit with the family. Tantrums? Yep, we've been there, had those. Persevere and the child will recognize that coming out of the chapel is much less interesting (nothing to look at or do!) than sitting in the pew with the family.
Another thing that helps - we use sign language to correct and communicate the need for quiet to our children when we're in a pew. This means I can look down the line of children and even though I can't reach one of my middle ones (who are still little!) I can tell them what I expect.
Once you reach 3 or 4 years old depending on the child's maturity we practice sitting quietly at home if they've had to be taken out during services at church that day.
Another round about way to help, we do family scripture study every day and it is done sitting and practicing being quiet if it is not your turn to read. We have actually done multiple practice sessions that are shorter in a day when we have a really little one needing this practice (age 1-2).
Hopefully those ideas and some from the others here give you some help!