Jump to content

Papyrus float comparison


Recommended Posts

Posted
DebugLog("" + 0.0001 as float) ;0.000100
DebugLog("" + (0.000000 < 0.0001)) ;False
DebugLog("" + Math.abs(0.00000)) ;0.000000
DebugLog("" + Math.abs(0.00000 - 0.00000)) ;0.000000
DebugLog("" + (Math.abs(0.00000 - 0.00000) < 0.0001)) ;False
DebugLog("" + (Math.abs(0.00000 - 0.00000) < 0.001)) ;TRUE

 

The log output is at the right after the comma.

But why? What I did wrong or is there anything to do with float precision?

 

Posted

Huh. First thought - "when is string conversion done - before or after comparison?.." So, if you put comparison inside an If and then put into log just predefined string based on true or false - does it also glitch?

Posted
On 7/15/2022 at 9:25 PM, DeWired said:

Huh. First thought - "when is string conversion done - before or after comparison?.." So, if you put comparison inside an If and then put into log just predefined string based on true or false - does it also glitch?

Yes it still does.

I changed float comparion into abs(a-b) < 0.001 now.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...