255 * 255 calculation


DevX Home    Today's Headlines   Articles Archive   Tip Bank   Forums   

Results 1 to 4 of 4

Thread: 255 * 255 calculation

  1. #1
    Miroslav St. Jeliaskoff Guest

    255 * 255 calculation

    Why I can't get this result?

    Label1.Capton = 255 * 255

    Isn't it strange? Atleast for me it is!

    Miroslav St. Jeliaskoff



  2. #2
    Bob Butler Guest

    Re: 255 * 255 calculation

    "Miroslav St. Jeliaskoff" <miroslav@earthling.net> wrote in message
    news:3d305378$1@10.1.10.29
    > Why I can't get this result?
    >
    > Label1.Capton = 255 * 255
    >
    > Isn't it strange? Atleast for me it is!


    255 fits into a 16-bit integer so that's what VB uses to store the constant.
    The largest data type in '255*255' is a 16-bit integer so that's what VB
    uses to hold the result.
    255*255> 32767 so there is an overflow error.

    To get around this promote at least one value to a larger type:

    Label1.Caption=CStr(255 * 255&)
    or
    Label1.Caption=CStr(255 * CLng(255))

    Note: the CStr is not required as VB will coerce the numeric result to a
    string to put it into the caption; I just prefer to make all data
    conversions explicit rather than relying on VB to guess what I mean.




  3. #3
    Miroslav St. Jeliaskoff Guest

    Re: 255 * 255 calculation

    Thanks Bob,
    I didn't know that. If I understand Atleast one of the arguments
    in that function has to be of a bigger type?

    Miroslav St. Jeliaskoff

    "Bob Butler" <butlerbob@earthlink.net> wrote in message
    news:3d30546a@10.1.10.29...
    > "Miroslav St. Jeliaskoff" <miroslav@earthling.net> wrote in

    message
    > news:3d305378$1@10.1.10.29
    > > Why I can't get this result?
    > >
    > > Label1.Capton = 255 * 255
    > >
    > > Isn't it strange? Atleast for me it is!

    >
    > 255 fits into a 16-bit integer so that's what VB uses to store

    the constant.
    > The largest data type in '255*255' is a 16-bit integer so that's

    what VB
    > uses to hold the result.
    > 255*255> 32767 so there is an overflow error.
    >
    > To get around this promote at least one value to a larger type:
    >
    > Label1.Caption=CStr(255 * 255&)
    > or
    > Label1.Caption=CStr(255 * CLng(255))
    >
    > Note: the CStr is not required as VB will coerce the numeric

    result to a
    > string to put it into the caption; I just prefer to make all

    data
    > conversions explicit rather than relying on VB to guess what I

    mean.
    >
    >
    >




  4. #4
    Randy Birch Guest

    Re: 255 * 255 calculation

    VB, in calculating the results to assign to x, uses a temporary variable. In
    doing so, it creates and uses an internal temporary working variable whose
    data type is the smallest possible one that would could contain without
    error either of the values being multiplied.

    In your case, because the both data types being multiplied are integers, VB
    uses an integer as the temporary working variable. As the result of the
    multiplication exceeds 32k (the limit for an integer), VB is forced to toss
    an overflow error (even though VB itself was responsible for creating the
    error in the first place!)

    To ensure that VB will use an internal temporary working variable of a data
    type sufficient to hold the result (as opposed to each item), you must
    assign a type-declaration character to one of the values in the expression.
    This, following the rules stated above, forces VB to use an internal
    temporary working variable of sufficient capability to hold the result, thus
    completing the action without error. For example:

    x = 33 * 1000&

    x will hold the correct result, since the expression indicated that 1000 was
    to be treated as a Long (even though the value 1000 fits inside an integer)
    and thus VB used a long to create its internal temporary working variable.
    However, be aware that the code:

    x = 33 * 1000 * 100&

    will still generate the error. In this case, despite the inclusion of the
    Long declaration character in the expression, VB must perform the
    calculation in parts. Since the expression did not contain brackets forcing
    one portion to be evaluated over another, VB attempts to compute the first
    part of the multiplication ( 33 * 1000 ) into its internal temporary working
    variable created it as an integer, again generating the error.

    The fix under these circumstances is, as above, to assign the long to the
    one of the first two values being evaluated largest value, or to bracket the
    equation to have VB perform the computation containing the long first:

    x = 33 * 1000& * 100
    x = 33 * (1000 * 100&)


    --

    Randy Birch
    MVP Visual Basic
    http://www.mvps.org/vbnet/
    Please respond only to the newsgroups so all can benefit.


    "Miroslav St. Jeliaskoff" <miroslav@earthling.net> wrote in message
    news:3d305378$1@10.1.10.29...
    > Why I can't get this result?
    >
    > Label1.Capton = 255 * 255
    >
    > Isn't it strange? Atleast for me it is!
    >
    > Miroslav St. Jeliaskoff
    >
    >




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center
 
 
FAQ
Latest Articles
Java
.NET
XML
Database
Enterprise
Questions? Contact us.
C++
Web Development
Wireless
Latest Tips
Open Source


   Development Centers

   -- Android Development Center
   -- Cloud Development Project Center
   -- HTML5 Development Center
   -- Windows Mobile Development Center