To test your battery, hold the meter probes on the battery posts while the engine is cranking... have a friend work the ignition switch for you. You want to see what the battery voltage is doing while the ignition switch is in the Start position. It should not drop below 10 volts after being cranked several seconds.
Without the engine running a fully charged conventional battery should measure 12.6 volts, a fully charged AGM battery will measure 12.8 volts.
And place the meter's probes on the battery posts themselves, not the connectors. If you get good voltage on the battery posts while the engine is cranking, move the meter's probes over to the connectors on the battery. If the voltage on the battery post is ok but not on the connectors, remove both connectors and wire brush the inside of them and the battery posts. Securely reinstall them so they are tight enough that they can't be moved around on the battery posts even with considerable hand pressure.
The very best way to test a battery is with a "load test" device which places a heavy load on the battery for 15 seconds while it measures the voltage. The voltage should not drop below 10 or so volts at the end of the 15 second test. Not all auto parts store sales people know how to properly conduct a load test, many don't know they need to hold the load on the battery for 15 seconds. Many will just hold the load on for a couple seconds which can give an erroneous good battery indication.