### Modulus Returning Incorrect Value

Posted:

**2021-Aug-06, 10:25 am**12 Aug 2012 18:11

Hi,

I need to find the numeric value for FPS from this format, e.g., 24000/1001 which is equivalent to 23.976. Here's a simple example script that is returning the incorrect mod of 977 rather than 976.

If you use a claculator, the result is actually: 23.97602397602398

Also, is there a way to tell the batch file how to treat the mod value, i.e., not to round up and precision? Thanks for any help.

----------------------------

#2 13 Aug 2012 15:13

I'm not sure if you did your math wrong, or if you misunderstand what the modulous operation does. But the batch script is giving the correct answers.

The result of integer divsion: 24000 / 1001 = 23

The remainder of integer division (modulous): 24000 %% 1001 = 977

Put another way, (23*1001)+977 = 24000

BTW - You don't need to expand variables when used in a SET /A computation, and you can perform multiple assignments with one command. You also don't require delayed expansion in this code.

----------------------------

#3 16 Aug 2012 04:21

Thanks for your reply and the tip.

Yes, you're right, it's correct. After I posted this, I actually did the math by hand using long division and had to laugh. lol

So here's my situation. I actually need the 976 from the calculated 23.97602397602398. So precision of 3 and essentially throwing out the 02397602398. This is used for video frames per second and it has to be accurate. Do you have any suggestions for how to get this part in a batch file? I know there are limits to what can be done here. I've been putting off moving to WHS due to this batch file is pretty large, but I may finally have to do that as I prefer C# anyway. TIA.

----------------------------

#4 16 Aug 2012 12:14

will yield only the first 3 characters.

See SET /?

Windows Shell Scripting and InstallShield

----------------------------

#5 16 Aug 2012 18:16

You can use the following to simulate floating point division with 3 decimal digits of precision (the 3rd decimal digit will be rounded up if the 4th is 5 or greater)

Dave Benham

Last edited by dbenham (16 Aug 2012 18:52)

----------------------------

#6 18 Aug 2012 00:25

Thanks so much all. Very much appreciated.

**AYColumbia**Hi,

I need to find the numeric value for FPS from this format, e.g., 24000/1001 which is equivalent to 23.976. Here's a simple example script that is returning the incorrect mod of 977 rather than 976.

Code: Select all

```
@echo off
setlocal EnableDelayedExpansion
set a=24000
set b=1001
set /a div=!a! / !b!
set /a mod=!a! %% !b!
echo !div! - !mod!
endlocal
```

Also, is there a way to tell the batch file how to treat the mod value, i.e., not to round up and precision? Thanks for any help.

----------------------------

#2 13 Aug 2012 15:13

**dbenham**I'm not sure if you did your math wrong, or if you misunderstand what the modulous operation does. But the batch script is giving the correct answers.

The result of integer divsion: 24000 / 1001 = 23

The remainder of integer division (modulous): 24000 %% 1001 = 977

Put another way, (23*1001)+977 = 24000

BTW - You don't need to expand variables when used in a SET /A computation, and you can perform multiple assignments with one command. You also don't require delayed expansion in this code.

Code: Select all

```
@echo off
setlocal
set a=24000
set b=1001
set /a div=a/b, mod=a%%b
echo %div% - %mod%
endlocal
```

#3 16 Aug 2012 04:21

**AYColumbia**Thanks for your reply and the tip.

Yes, you're right, it's correct. After I posted this, I actually did the math by hand using long division and had to laugh. lol

So here's my situation. I actually need the 976 from the calculated 23.97602397602398. So precision of 3 and essentially throwing out the 02397602398. This is used for video frames per second and it has to be accurate. Do you have any suggestions for how to get this part in a batch file? I know there are limits to what can be done here. I've been putting off moving to WHS due to this batch file is pretty large, but I may finally have to do that as I prefer C# anyway. TIA.

----------------------------

#4 16 Aug 2012 12:14

**RG**Code: Select all

`set mod=%mod:~0,3%`

See SET /?

Windows Shell Scripting and InstallShield

----------------------------

#5 16 Aug 2012 18:16

**dbenham**You can use the following to simulate floating point division with 3 decimal digits of precision (the 3rd decimal digit will be rounded up if the 4th is 5 or greater)

Code: Select all

```
@echo off
setlocal
set a=24000
set b=1001
set /a "whole=a/b, fract=(a%%b*10000/b+5)/10"
set "fract=000%fract%"
set "fract=%fract:~-3%"
echo %a%/%b% = %whole%.%fract%
```

Dave Benham

Last edited by dbenham (16 Aug 2012 18:52)

----------------------------

#6 18 Aug 2012 00:25

**AYColumbia**Thanks so much all. Very much appreciated.