Questions tagged [decimal]
Decimal is the name for our common base-ten numeral system. It may also refer to non-integer values expressed with a decimal point.
decimal
7,628
questions
0
votes
0
answers
8
views
Kendo Field with 4 decimal places No decimal rounding
The problem occurs when the value is entered and it is automatically converted to decimals Rond!
$("#grid").kendoGrid({
dataSource: dataSource,
pageable: true,...
- 1
0
votes
1
answer
23
views
Java using OJDBC8 driver are dropping decimal values when inserting into column defined as NUMBER in Oracle(19c) [closed]
Java8 using OJDBC8 driver are dropping decimal values when inserting into column defined as NUMBER.
This does not occur when using OJDBC7.jar. The client is using Oracle 19c database which requires ...
0
votes
1
answer
37
views
How to reduce decimal places to just 2?
I have this query in BigQuery
SELECT Id,
AVG(VeryActiveMinutes) AS avg_very_active,
SUM(VeryActiveMinutes) AS total_very_active,
SUM(SedentaryMinutes) AS total_sedentary
FROM `...
0
votes
0
answers
18
views
Changing no. decimal points displayed for quantile values on boxplot [ggboxplot() + stat_summary(.."%1.1f"..)]
I am using R (v4.4.2).
CONTEXT:
I have a data frame, called "soil moisture", with relevant variables of Substrate (categorical values, "Dung" and "Sand") and SM (...
- 11
1
vote
0
answers
73
views
Is Decimal a lot slower in MacOS Swift than Decimal in Python?
[update]fyi: I did try running the same test below with GMP lib in Swift and it is about 10x faster than Swift Decimal which I imagine the upper bound would be. Still confused why Python Decimal would ...
- 105
2
votes
1
answer
63
views
Return generic to its discrete type
I run a bunch of calculations using Decimal and Double. Without generics there'd be a lot of repetitive code. There are places I eventually need the Decimal in a more native format but I can't seem to ...
- 105
0
votes
1
answer
52
views
How to convert Hex to Dec in Codesys ST?
I'm using Codesys V3 ST language
I'm writing a code to communicate motor controller unit which uses CANOpen protocol
I want to use CAN raw messages to communicate
I put CAN bus messages bytes in an ...
- 35
-4
votes
1
answer
61
views
Converting a HEX color value to RGB in JavaScript
I have this code:
let hexInt = parseInt(hex, 16);
let r = (hexInt >> 16) & 255;
let g = (hexInt >> 8) & 255;
let b = hexInt & 255;
I know that a HEX digit requires 4 bits to ...
- 1,483
0
votes
1
answer
79
views
How to make python use more than 15 decimals in calculations?
In this question I made a claim that this construction of pentagons will always converge.
I drew the first 50 iteration using Geogebra here. However, zooming in on the 50th iteration resulted in ...
- 95
0
votes
0
answers
53
views
Decimal.TryParse in c# does not return decimal [duplicate]
This is the part of my code:
decimal metricval = 0;
if (decimal.TryParse(attrVal, out metricval))
{
decimal multiplyBy = getTheMultiply("lTogal");
...
- 357
2
votes
1
answer
38
views
Python Set Context precision for Decimal field
from decimal import Decimal, setcontext, getcontext
class MyNewSerializer(serializers.Serializer):
total_value_base = serializers.SerializerMethodField()
total_outgoing_value_base = ...
- 880
-2
votes
0
answers
36
views
i convert integer as hex to decimal in PHP but it gives me float, how to solve it? [duplicate]
when I try to convert this Hex: 780000465257A7054F5180E46C000100004652 to a decimal number, it should give me this decimal value: 2676089517297180626710225817179225498977322578,
you can try that using ...
- 71
0
votes
2
answers
39
views
Extract(year from column) yields year with one decimal in SQL, why?
I used this to extract year from datetime:
select extract(year from orders.OrderDate)
It gave me year with one decimal. e.g. 2019.0
Why is this? Any feedback would be appreciated!
I also tried:
YEAR(...
- 1
0
votes
0
answers
75
views
C# modulo operation with decimal yields inconclusive result
I have a very quick question, based on the following C# code:
decimal _sampleCounter = 2100m;
decimal wrong = _sampleCounter % (30m / 2.3m);
decimal correct = 2100m % (30m / 2.3m);
Console.WriteLine(...
- 1
0
votes
1
answer
45
views
Swift generic Decimal divide operator missing
I was trying to come up with some code using generics to switch between Double and Decimal and setup this simple example using all the "Conforms To" relationships in Decimal which is a ...
- 105